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PREFACE 


This  study  was  conducted  at  the  request  of  the  Naval  Sea 
Systems  Command's  Cost  Estimating  and  Analysis  Division, 
Code  017.  Funding  was  provided  under  the  Naval  Postgraduate 
School's  direct  funding  allotment,  Project  Code  N4P1.  The 
objectives  of  this  study  were  stated  in  the  proposal  document: 
Determine  whether  theory  drawn  from  related  disciplines 
holds  promise  for  improving  our  understanding  of  how 
SOA  [ State-of-the-Art ]  extensions  relate  to  development 
and  production  costs  of  military  electronic  components. 

If  so,  attempt  to  build  a  quantifiable  relationship. 
(Note:  the  accomplishment  of  this  objective  will 
require  specification  of  measurements  for  SOA 
extensions — to  be  drawn  from  measurement  theory.) 

This  final  report,  along  with  copies  of  the  data  used,  is 
submitted  in  fulfillment  of  the  agreement.  The  report  is 
releasable,  but  the  supplementary  data  will  be  distributed  only 
to  DoD  agencies  by  request. 

The  author  wishes  to  thank  several  people  who  made  this 
undertaking  possible.  Most  notable  among  them  were  Michael 
Hammes  (Director,  017),  Jim  Herd  (now  with  NCA),  Bill  Broocke, 
Scott  Endres  and  John  Maurer.  Finally,  Captain  Blain  Webber 
(USAF)  was  especially  helpful  in  providing  and  interpreting 
satellite  data. 


il 


EXECUTIVE  SUMMARY 


When  the  various  components  of  the  Department  of  Defense 
(DoD)  enter  into  contractual  agreements  calling  for  extensions  of 
the  State-of-the-Art  (SOA),  uncertainty  generally  requires  that 
some  variant  of  cost-plus  contracting  be  employed.  Therefore,  it 
is  important  that  DoD  possess  a  highly  developed  ability  to 
estimate  the  likely  cost  of  achieving  the  desired  technological 
advance . 

This  study  began  by  examining  the  techniques  currently 
available  for  costing  SOA  extension  contracts  by  surveying  the 
literature  dealing  explicitly  with  SOA  measurement  and  costing. 
SOA  measurement  strategies  usually  employ  regression  to  combine 
design  variables  in  an  effort  to  produce  a  single  measure  of  the 
level  of  technology  imbedded  in  a  given  system.  The  typical 
dependent  variable  is  the  "year  of  technology". 

The  relationship  between  the  scale  of  an  SOA  extension  and 
development  cost  has  also  been  studied,  but  the  findings  from 
these  studies  have  been  disappointing.  Cost  estimates  made  early 
in  the  life  of  a  project  have  been  plagued  with  error.  The  focus 
of  this  study  has  therefore  been  to  develop  a  cost  estimating 
model  that  is  demonstrably  workable  for  both  prediction  and  cost 
control  uses. 

In  selecting  the  variables  to  be  used  to  measure  the 
technology  embedded  in  a  type  of  system  to  be  studied,  there  is 
no  substitute  for  expert  judgement.  Experts  must  bear  in  mind 


that  the  variables  selected  must  be  influenced  by  the  engineering 
development  decisions.  They  should  choose  characteristics  which 
are  go  al  s  of  the  design  process.  In  addition,  the  variables 
should  be  specified  so  that  increasing  values  correspond  to 
greater  technical  difficulty.  Finally,  the  variable  values 
should  be  ascertainable  during  the  early  decision-making  stages 
of  the  system  life-cycle. 

In  the  present  study  18  variables  or  composite  variables 
that  could  be  used  to  describe  satellite  technology  were 
identified.  There  was  unclouded  discussion  and  the  results 
obtained  represented  consensus. 

A  factor  analysis  was  then  run.  In  its  final  fora,  11  of 
the  18  variables  were  factored.  They  clustered  very  nicely  onto 
four  factors  with  81. 7X  of  the  variance  explained. 

The  next  step  waa  to  calculate  factor  acoras  for  each  of  the 
18  systems  in  the  data  act.  Note  that  factor  scores  represent 
blends  of  parameters  affecting  the  level  of  system  technology. 

The  ellipsoid  model  was  then  used  to  determine  the  level  of 
technology  embodied  in  each  satellite.  This  technology  measure, 
the  radial  distance  from  the  origin,  is  a  second-order  function 
of  the  four  factor  scores. 

The  ead  purpose  of  developing  SOA  measures  is  to  facilitate 
prediction  of  the  cost  of  developing  new  technological  systems, 
which  is  a  necessary  initial  step  in  any  attempt  to  control  such 
costs.  The  next  step  was  therefore  to  search  for  statistical 
associations  between  (1)  the  degree  to  which  a  system's 


technology  is  extended,  and  (2)  the  level  of  activity  required  to 
bring  this  extension  about. 


The  technological  objective  of  the  project  and  the  closest 
ex  is  ti  ng  technology  were  used  to  identify  the  development  task 
by  referencing  the  technological  distance  separating  the  two. 
Several  more  detailed  measurement  concepts  were  then  developed. 
The  re ac h  measures  the  total  technological  complexity,  or  the 
overall  ambition  of  the  project.  Advance  represents  the 
"invention"  aspects  of  the  development — the  "true"  SOA  progress 
required.  The  redesign  portion  represents  a  movement  parallel  to 
an  old  SOA  surface. 

The  first  hypothesis  tested  was  that  the  difficulty  of  the 
development  task,  as  measured  by  the  time  required  for  its 
completion .  is  a  function  of  the  three  measures  of  technological 
spread.  The  result  was, 


Time  -  52.86 

+  218.93  Advance 

-  34.28 

Redesign  - 

17.37  Reach 

t  statistics 

(3.69) 

(1.45) 

(0.47) 

Significance 

.001 

.085 

.322 

Variance 

explained  (R2) 

.791 

Adjusted 

R2 

.728 

Standard 

error  of  the  estimate 

8.745 

The  regression  is  highly  significant.  Advance  is  by  far  the  most 
important  determinant  of  development  time.  Neither  redesign  nor 


smooth  fuoctioQ  of  development  time.  If  a  program  drags  on 
beyond  its  intended  completion  date,  it  becomes  relatively  more 
costly  to  compress  the  required  accomplishment  into  an 
increasingly  abbreviated  time  horizon.  This  suggests  that  there 
is  a  "natural"  project  time,  and  that  the  residuals  from  this 
natural  time  may  influence  cost.  Again,  the  multiple  regression 
produced  good  results, 

Cost  «  -  61357  +  4793.1  Predicted  Time  +  7391.4  Residual 


t  statistics 

(3.12) 

(2.47) 

Significance 

.004 

.013 

Variance 

explained  (R2) 

.590 

Adjusted 

R2 

.516 

Standard 

error  of  the  estimate 

82647 

The  most  basic  task  in  cost  control  is  to  explain  variances 
between  predicted  and  actual  costs.  The  regressions  provided  a 
basis  for  doing  so. 

First,  Advance,  Redesign  and  Reach  were  used  in  the  "Time" 
regression  to  predict  the  time  that  would  be  required  for  the 
system's  development.  Then  the  Predicted  Time  was  input  to  the 
"Cost"  regression  (with  the  Residual  set  to  zero)  to  provide  an 
ex  ante  prediction  of  development  cost. 

Next  the  actual  time  for  the  project  was  compared  to  the 
nredicted  time  to  determine  the  Residual.  The  Cost  regression 


was  then  used  again  —  this  time  to  calculate  a  new  cost  estimate 
considering  the  residual  time  for  the  project.  The  difference 
between  the  ex  ante  cost  estimate  and  the  cost  estimate  based  on 
the  project's  actual  time  is  the  "Variance  Due  to  Time"  —  the 
portion  of  the  total  variance  that  can  be  attributed  to  the  cost 
consequences  of  time  delays. 

Actual  Cost  is  compared  with  the  cost  estimate  based  on 
actual  time  to  determine  a  "Cost  Control  Variance".  This 
variance  indicates  the  quality  of  cost  control  for  the  project. 

The  fundamentals  developed  in  this  study  provide  a  workable 
methodology  for  measuring  the  level  of  technology  embodied  in 
complex  systems,  and  for  measuring  the  degree  of  advance 
represented  by  the  technological  characteristics  of  new  systems 
compared  with  old.  This  capability  can  provide  information  that 
may  be  useful  for  cost  control. 

The  requisite  data  used  in  implementing  this  procedure  have 
a  great  deal  to  do  with  the  quality  of  the  results  achieved.  If 
the  benefits  of  good  cost  control  are  to  be  obtained  it  is 
absolutely  essential  to  have  a  relevant  and  complete  data  base. 
It  is  necessary  to  have  at  least  four  valid  observations 
(systems)  in  the  data  base  for  every  factor  that  is  identified 
through  the  factor  analysis  procedure. 

The  chief  recommendation  of  this  study  is  that  DoD 
components  who  engage  in  technology-extension  contracting 
carefully  design  and  maintain  high-quality  data  bases  on  systems 
that  are  likely  to  be  targets  for  further  development. 
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Chapter  1 


COSTING  SOA  EXTENSION  CONTRACTS:  A  LITERATURE  REVIEW 

i 

INTRODUCTION 

Various  components  of  the  Department  of  Defense  (DoD), 
including  all  the  services,  enter  into  contractual  agreements 
calling  for  development  of  new  technology,  or  for  extensions  of 
the  S t a t e - o f - t h e - A r t  (SOA).  Occasionally  the  required 
development  task  can  be  sufficiently  defined  to  allow  use  of  a 
fixed  price  contract.  More  commonly,  though,  uncertainty 
requires  that  some  variant  of  cost  plus  contracting  be  employed. 
In  these  circumstances  the  services  must  somehow  estimate  the 
likely  cost  of  achieving  the  desired  technological  advance. 

This  chapter  examines  the  techniques  currently  available  for 
costing  SOA  extension  contracts  by  surveying  the  more  important 
literature  dealing  explicitly  with  SOA  measurement  and  costing. 
The  chapter  then  looks  at  SOA  costing  trends  and  discusses  the 
new  challenges  that  lie  ahead. 

Background 

The  cost  of  developing  an  SOA  extension  relates,  presumably, 
to  the  scale  of  the  undertaking.  That  is,  it  should  be  more 
costly  to  extend  an  existing  SOA  ten-fold  than  it  would  be  to 


make  only  a  modest  improvement.  Therefore,  the  task  of  SOA  cost 
estimating  must  be  reducible  to  (1)  measuring  the  degree  of  SOA 
extension  represented  by  the  contract  and  (2)  establishing  a 
relationship  between  degrees  of  extension  and  development  cost. 

Measuring  levels  of  and  extensions  in  SOA  have  been  the 
subjects  of  much  research.  (The  more  important  of  these  studies 
will  be  discussed  in  the  next  section.)  The  usual  approach  has 
been  to  identify  a  limited  number  of  design  variables  or 
performance  parameters  that  represent  the  overall  level  of 
technology  embodied  in  the  particular  system  or  assembly  under 
study.  Examples  might  include:  resolution  power  and  size  for 
sensor  arrays;  computational  speed  and  memory  size  for  computers; 
specific  impulse  and  burn  time  for  rocket  motors. 

Often  there  are  potential  trade-offs  among  performance 
variables  within  a  particular  SOA,  such  as  with  speed  and  useful 
load  for  aircraft.  SOA  assessment  strategies  frequently  employ 
mathematical  weighting  techniques,  such  as  regression,  to  combine 
design  variables  in  an  effort  to  produce  a  single  measure  of  the 
SOA  for  a  given  system.  One  commonly  (perhaps  too  commonly)  used 
measure  is  the  "year  of  technology". 

The  relationship  between  development  cost  and  the  scale  of 
an  SOA  extension  has  also  been  extensively  studied.  However, 
findings  from  these  studies  have,  for  the  most  part,  been 
disappointing.  Cost  estimates  made  early  in  the  life  of  s 
project  have  been  plagued  with  error.  In  the  latter  stages  (when 
it  may  be  too  late  to  avoid  heavy  cost  overruns)  SOA  extension 


measures  are  usually  irrelevant;  the  program  is  far  enough  along 
to  allow  more  accurate  bottom-up  costing  at  a  highly  detailed 
level  of  the  Work  Breakdown  Structure  (WBS). 


By  chronologically  examining  the  development  of  SOA 
extension  cost  estimating  techniques,  one  can  discern  a  pattern 
that  may  be  useful  in  guiding  the  development  of  further 
improvements.  Specifically,  this  analysis  will  suggest  that  the 
derivation  of  empirically  based  parametric  models  with  somewhat 
more  detailed  input  would  produce  superior  results  due  to 
expansion  of  the  relevant  data  base  and  to  the  infusion  of 
additional  methodological  rigor.  Thus  our  next  challenge  will  be 
to  prescribe  a  disciplined  process  for  cost  estimators  to  use  in 
deriving  appropriate  models  at  the  system,  sub-system  and 
component  levels  of  the  SOA  extension. 

COSTING  LITERATURE 

Literature  on  Measuring  SOA 

Dodson  and  Graver,  1969.  While  SOA  has  been  measured  using 
numerous  methods  for  many  years,  Dodson  and  Graver  reported  an 
important  theoretical  advance  in  1969.1  Their  innovation  was  to 
make  use  of  convex  (ellipsoidal)  h yp er su r f ac es  to  represent 
particular  levels  of  SOA.  In  order  to  better  understand  the 
approach,  consider  that  three  steps  must  be  taken: 

1.  An  operational  definition  of  SOA  must  be  specified. 


2.  N  SOA-determining  parameters  must  be  specified  for  the 
kinds  of  systems  or  subsyjtems  under  exaaination. 

3.  An  N_-d  iaensional  ellipsoid  must  then  be  fit  to  the 
parameter  measurements  of  an  SOA-representative  group  of  systems 
or  subsystems. 

Dodson  and  Graver  addressed  the  first  step  by  defining  SOA 
as  follows; 

In  concept,  SOA  might  be  defined  as  the  highest  degree 
of  technical  accompli shaent  that  could  be  achieved  at 
any  point  in  tiae.  In  practice,  however,  it  would  be 
impossible  to  reconstruct  objectively  what  performance 
characteristics  could  have  been  realized  at  tiaea  in 
-  the  past.  On  the  other  hand,  performance  and  design 
characteristics  that  were  achieved  can  be  determined 
objectively.  This  suggests  adoption  of  "state  of  the 
art"  as  defined  by  the  physical  and  performance 
characteristics  of  development  programs  completed 
during  the  tiae  period  in  question. 2 
Therefore,  Dodson  and  Graver  stipulate  recently  implemented 
technology  as  representing  the  current  SOA. 

Since  no  single  measures  of  SOA  have  yet  been  developed,  N 
SOA-deteralnlng  parameters  must  be  specified  in  such  a  way  as  to 
collectively  describe  the  level  of  techi^logy  embodied  in  the 
kinds  of  systems  or  subsystems  under  examination.  Dodson  and 
Graver  stated  three  guidelines  to  use  in  selecting  parameters  to 
represent  SOA.  First, 
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The  selection  of  SOA-determining  characteristics  should 
be  concentrated  on  those  that  are  at  least  partially 
influenced  by  engineering  development  decisions.  The 
individual  characteristics  .  .  .  represent  constraints 

upon  the  achievement  of  other  design  characteristics 
which  are  —  to  a  degree  —  goals  of  design. 3 
Second , 

.  .  .  Characteristics  should  be  specified  so  that 

increasing  values  of  the  characteristics  correspond  to 
greater  technical  difficulty.4 
Finally , 

.  .  The  characteristics  should  be  aaong  those  that 

are  typically  ascertainable  during  the  early  decision¬ 
making  stages  of  the  system  life-cycle. ^ 

This  last  guideline  is  particularly  important  if  the  final 
analytical  output  is  to  be  useful  when  R&D  funding  decisions  are 
being  made,  or  at  the  early  feasibility  study  stage. 

Once  the  N_  SOA-deteraining  parameters  have  been  specified 
and  measured  across  an  appropriate  sample,  Dodson  snd  Graver 
theorized  that  an  SOA-descri bing  ellipsoid  could  be  fit  to  the 
data  by  minimizing  squared  proportional  deviations  (using  scale 
free  proportional  measurements  from  the  origin).  The  approach 
is  best  understood  by  envisioning  the  process  in  only  two 
dimensions.  (See  Figure  1.1  on  the  next  page.) 
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Figure  1.1  Dodson  and  Graver's  Ellipsoid  Approach 

Two  systea  attributes  have  been  scaled  on  the  two  axes.  The 
elliptical  curve  represents  a  particular  level  of  technology  or 
SOA.  If  aission  requireaenta  call  for  aore  of  one  attribute  and 
less  of  another,  the  need  could  be  act  by  aovlng  along  the  curve. 
Note  that  SOA  is  not  increased.  Basically,  the  ellipsoid  is  an 
iso-SOA  curve. 

The  SOA  curve  exhibits  the  econoaic  property  of  diainishing 
returns.  In  order  to  obtain  aore  of  one  attribute,  ever- 
increasing  quantities  of  the  other  Bust  be  given  up.  This  is  a 
coaaon  phenoaenon  in  design. 

Dodson  and  Graver  argue  that  (in  N-space)  a  new 
hypersurface  is  required  for  each  SOA.  The  suggested  inference 
is  that  the  level  of  SOA  advance  eabodied  in  a  systea  with  a 
particular  set  of  attributes  is  the  radial  distance  of  the 
desired  point  froa  the  aost  current  ellipsoid. 

Dodson.  1970.  In  1970  Dodson  published  an  oft-cited 
article  reporting  and  expanding  upon  the  results  of  his  work 


with  Graver. 6  In  this  article  he  went  on  to  suggest  that  the 
ellipsoid  aodel  would  be  appropriate  only  when  all  system 
attributes  approach  finite  upper  bounds: 

When  the  dimensions  include  one  or  more  terms  which  do 
not  approach  a  finite  upper  bound,  the  fora  is  planar. 

Thus,  the  choice  depends  upon  the  nature  of  the  SOA- 
determining  parameters. ? 

Next,  he  gathered  data  on  seven  missile  propulsion  subsystems 
with  development  completion  dates  between  1951  and  1955  and  fit  a 
f  o  ur -d  im  en  si  onal ,  linear  hypersurface  to  the  data.  The  result 
is  a  simple  equation  defining  the  prevalent  SOA  during  that  time 
period : 


Xj  X2  X^ 

245.9  *  1372  +  117.7  +  07  "  1 


where  X^  ■  delivered  specific  impulse  at  sea  level,  in  seconds 
X2  ■  propellant  weight/total  motor  weight  (including 
propellant ) 

X ^  ”  length-to-diaaeter  ratio  of  aotor  case 
X^  -  1/burn  tiae,  in  seconds.** 

To  test  this  aodel  Dodson  gathered  additional  data  on  a 
number  of  rocket  motors  developed  between  1957  and  1961  and 
calculated  SOA  values  for  each  using  the  same  equation.  Points 
representing  subsequent  systems  were  found  to  lie  increasingly 
above  the  surface  established  for  the  1951-55  time  interval. 

Alexander  and  Nelson,  1972.  Alexander  and  Nelson's  1972 
RAND  report  began  by  stating  an  important  limitation  to  the 
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usefulness  of  quantitative  models  for  measuring  SOA.  The 
limitation  is  that  the  progression  of  technology  must  conform  to 
an  assumption  of  "continuity"; 

Continuity  exists  if  two  devices  that  appear  at 
different  times  can  be  characterized  by  the  same  set 
of  parameters.  Continuity  also  requires  that 
subsequent  development  can  begin  where  prior 
development  ended.  .  .  .  Basic  research  and  invention 

are  excluded  from  our  purview  .  .  .  the  output  of  such 

activity  is  unique,  unpredictable,  and  unspecif iable. 9 
This  important  limitation  qualifies  the  use  of  parametric  SOA 
measurement  models  under  evolutionary  conditions  exclusively. 

Like  Dodson,  Alexander  and  Nelson  theorize  the  existence  of 
a  curvilinear  relationship  among  technological  parameters.  The 
following  illustration  demonstrates  the  posited  association 
between  thrust  and  weight  for  aircraft  turbine  engines.  The 
curve  labeled  t^  represents  the  level  of  technology  existent  at 
time  1_.  Technology  characteristic  of  time  0  required  more  weight 
for  an  engine  of  equivalent  thrust,  while  time  2_  technology 
enabled  an  engine  with  the  same  thrust  to  be  lighter.  Note  the 
diminishing  returns,  in  terms  of  thrust,  to  increasing  the  weight 
of  an  engine.  The  design  objective  is  lightness,  of  course:  if 
we  were  to  reverse  the  weight  axis  we  would  see  a  Dodson-like 
ellipsoidal  shape. 
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Figure  1.2  Alexander  and  Nelson's  Turbine  Engine  Example 


But,  also  like  Dodson,  Alexander  and  Nelson  used  linear  methods 
to  analyze  their  empirical  data. 

The  general  form  of  the  model  selected  was, 

t  -  F(Plf  .  .  .  ,  Pn) 

where  t_  is  the  point  in  time  one  would  expect  the  described 
technology  to  appear  and  the  P^s  denote  the  performance 
parameters  used  to  specify  the  technology. 

Data  describing  47  engines  developed  from  the  early  1940s  to 
the  late  1960s  were  subjected  to  analysis.  The  best  results  were 
produced  by  a  semi-logarithmic  form  of  the  model. 

Tech  •  -1187.5  +  156  In  Temp  +  18.8  ljn  Thrust 

-  26.5  In  Weight  -  20.6  In  SFC  ♦  11.7  In  Q  +  13.0  Prop 

where , 

Tech  *  Technology  index;  model  qualification  test  date; 
quarters,  4th  quarter  1942  equals  one. 
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Teap  •  Turbine  inlet  temperature;  Degrees  Rankine. 

Thrust  ■  Military  sea  level  static  thrust;  lb.  or  ESHP 
(equivalent  static  horsepower)  if  turboprop. 

Weight  *  Engine  weight;  lb. 

SFC  •  Specific  fuel  consumption  at  military  sea  level  static 
thrust;  (lb/hr)/lb  thrust. 

2 

Q  •  Maximum  dynamic  pressure;  Ib/ft  . 

Prop  ■  Dummy  variable;  one  if  turboprop,  zero  otherwise. ^ 

2 

The  regression  had  an  unadjusted  R  of  .903  and  a  standard  error 
of  9.6  quarters.  Alexander  and  Nelson  expressed  disappointment 
over  the  relatively  large  standard  error  saying,  "The  equation 
therefore  cannot  be  used  for  making  fine  diatinctions.  ^ 
Nevertheless,  they  plot  expected  versus  actual  time  and  say. 

The  45*  line  is  therefore  defined  as  the  average  trend 
or  expected  date  of  technology  over  the  period.  Points 
plotted  above  the  45*  line  represent  engines  "ahead  of 
their  time":  .  .  .  points  below  the  line  are  "late”  or 

"conservative"  developments . ^ 

As  an  aside,  there  is  an  interesting  section  in  the 
Alexander  and  Nelson  report  comparing  the  advance  of  U.S.  jet 
engine  technology  with  Soviet  engines.  They  conclude  the 
Soviets  were  ahead  of  the  U.S.  in  the  1940s,  but  that  the  Soviet 
lead  disappeared  in  the  early  1950s.  "Since  that  time  the  gap 
between  the  two  technologies  appears  to  have  widened."13  (With 
the  U.S.  pulling  steadily  ahead.) 
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Gordon  and  Munson,  1981.  This  paper  made  use  of  two 
general  forms  of  SOA  equations, 


and , 


where 


S  -  K.V.  +  K0V0  +  .  .  .  +  K  V 
11  L  L  n  n 


S  "  Vltl[2V2  +  K3V3  *  *  *  *  +  KnVnJ  (2) 

S  -  State  of  the  Art 

•  A  judgementally  or  statistically  assigned  weight 
V  ■  The  value  of  the  ith  technology-describing 
variable  expressed  as  a  decimal  fraction  of  the 
largest  value  of  that  variable  in  the  data  set. 


The  first  version  of  the  model.  Equation  1,  is  familiar.  The 
second  version  is  a  multiplicative  form  intended  for  use  when  one 
parameter  must  be  present.** 

There  are  two  data-based  examples  in  the  Gordon  and  Munson 
paper.  The  first  is  an  example  of  SOA  measurement  of  a  large 
sample  of  antibiotics  using  Equation  2,  where, 

■  Minimum  Inhibitory  Concentration. 

In  this  portion  of  the  study  the  K^s  were  established,  using 
subjective  methods,  by  "the  study  team  and  its  consultants.”*^ 

The  second  example  used  Equation  1  to  study  computers 
introduced  to  the  market  from  1951  through  1980.  In  the  first 
part  of  this  example  the  E^s  were  also  established 
subjectively.*^  The  variables  were, 


Vj  ■  Computer  speed  (operations/second) 

?2  ■  Cost  of  computation  (^nerations/dollar ) 

■  Maximum  memory  size  (kilobytes) 

Kj  -  0.5,  I2  -  0.3,  K3  -  0.2 

Later  in  the  article  Gordon  and  Munson  discussed 
statistically  fitting  S-curves  to  data,  to  demonstrate  the 
typical  S-shaped  growth  cycle  of  technology.  Several  versions  of 
S-curves  were  discussed,  but  the  one  fitted  to  the  computer 
example  (by  numerical  methods)  was  of  the  form, 

SOA  ■  j  (1  +  tanh  A  (t  -  t0)] 

where,  L  *  the  theoretical  upper  limit  of  the  variable 
A  ■  a  constant  that  depends  on  the  slope  at  the 
inflection  point 

t  -  time  of  technology  (as  in  Alexander  and  Nelson) 
t0  *  the  time  associated  with  the  inflection  point 

The  result  valued  Ij  at  0.14,  K2  at  0.38  and  at  0.49. *7 

As  an  aside,  Gordon  and  Munson  did  not  discuss  the 
reliability  of  their  models.  But  they  read  the  calculated  value 
of  the  L  parameter  for  the  S-curve  as  an  estimate  that  the 
ultimate  limit  to  computing  technology  will  be  "an  improvement  of 
about  25  percent  over  the  performance  of  the  IBM  3033. "1®  In 
fact,  today's  leading  computers  are  about  100  times  fasten,  are 
roughly  comparable  as  to  cost  per  computation,  and  contain  about 
60  times  more  storage  capacity  than  the  IBM  3033.  Using  the  two 
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versions  of  Gordon  and  Munson’s  model  for  computers  (two  sets  of 
^s),  the  implication  is  that  performance  has  in  fact  increased 
by  from  4340  to  6200  percent  over  that  of  the  IBM  3033  in  less 
than  a  decade! 

Another  contribution  made  by  Gordon  and  Munson  was  to 
suggest  the  use  of  factor  analysis  as  a  method  for  grouping 
proposed  technology  parameters  into  clusters  that  have  similar 
behavior  or  influence  on  S0A.19  a  possible  advantage  of  this 
procedure  is  that  it  might  allow  initial  inclusion  of  many  more 
parameters,  but  subsequent  stages  of  the  analysis  could  be  based 
on  a  more  limited  set  of  clusters  rather  than  on  the  many 
parameters  who's  information  content  had  been  so  aggregated. 
The  favorable  result  is 'parsimony .  However,  a  disadvantage  they 
see  in  factor  analysis  is  some  loss  of  intuitive  appeal  and 
increased,  perhaps  unnecessary,  complications  in  modeling. 

Knight.  1985.  Knight  examined  120  computer  systems 
introduced  between  1963  and  1979.  His  study  made  two  important 
contributions:  he  distinguished  sharply  between  functional  and 
structural  measures  of  technology,  while  developing  a 
relationship  between  the  two;  and  he  examined  the  movement  of  a 
functional  measure  of  computer  technology  over  time. 2® 

Knight  defined  the  concept  of  functional  measures  of 
computer  technology  as,  "The  capability  of  each  system  to  perform 

tt  2  1 

its  intended  tasks.  The  measurements  he  selected  to  indicate 

how  well  computers  perform  their  intended  tasks  were, 


Computing  power  in 

The  cost  of  the  in 

computing  equipment 

Computing  reliability  in 

Communications  cost  in 


Operations  per  second 

Seconds  per  dollar  of 
equipment  cost 

Mean  number  of  operations 
between  failures 

Seconds  per  dollar  of 
communications  cost 


The  structural  characteristics  of  a  system  are  those 
describing  the  technical  make-up  of  the  machine  itself.  Knight 
provides  a  detailed  description  of  a  computer’s  structure  which 
was  based  on  specifications  articulated  earlier  by  Burke, 
Goldstine  and  von  Neumann. 22  The  designs  are  specified  as 
follows : 


Memory  unit  *■>  Primary  internal  memory 

Secondary  internal  memory 
Dead  storage 

Arithmetic  unit  -»>  Operations: 

Arithmetic  registers 
Fixed  point  arithmetic 
Branching 

Instruction  alteration 
Number  system 
Arithmetic  mode 

Control  unit  «■>  The  instruction  source 

Order  of  operations 
Communication  to  and  from  the 
computer 

Timing  of  operations 
Error  check 

Input-out  unit  ■■>  Primary  input-output  system 

Secondary  input-output  system 
Tertiary  output  system 

Knight  carefully  pointed  out  that  several  computers  with 

different  structural  characteristics  can  be  functionally 
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equivalent,  but  that  the  reverse  does  not  hold.  That  is, 
structure  determines  functionality,  but  functionality  does  not 
necessarily  determine  structure. 23 

Knight's  objective  was  to  determine  the  functional 
characteristics  of  a  large  sample  of  computers  and  to  trace  the 
advance  of  those  characteristics  through  time.  He  began  by 
stating  that  reliability  and  communications  cost  were  too 
in st al la ti on -s pe ci f i c  to  be  determinable  by  computer,  so  his 
focus  was  limited  to  power  and  equipment  cost  (or,  more 
accurately,  computer  equipment  economy  since  the  variable  is 
calculated  by  dividing  usable  seconds  of  system  operations  by 
lease  price). 

Knight  also  stated  that  computing  power  is  software- 
dependent  to  a  significant  degree.  Therefore,  he  developed  a 
model  to  convert  computer  structure,  which  is  more  directly 
measurable,  into  a  measure  of  computing  power.  (Remember  that 
structure  determines  functionality.) 

As  a  final  step,  Knight  determined  the  relationship  between 
computer  equipment  economy  and  computing  power  through  time  (with 
both  variables  expressed  in  log  form).  His  work  can  be 
summarized  in  the  following  display.  (See  Figure  1.3  on  the  next 
page.)  The  computers  introduced  to  the  market  during  the  1963-64 
time  period  were  modest  in  terms  of  economy,  and  can  be 
characterized  as  low  power.  Computers  from  the  1979  era  were 
considerably  higher  in  power  and  more  variable  in  their  economy. 
Overall,  there  appears  to  be  a  roughly  linear,  negative 


Computing  Economy  vs.  Power 


In  Log— Log  Form 


Inc  waring  Computing  Poir 

□  1963-64  ♦  Intenrening  O  1979 

Figure  1.3  Summary  of  Knight's  Work 


relationship  between  the  two.  A  word  of  caution  is  in  order, 
however.  While  "seconds  per  dollar"  is  certainly  an  attribute  of 
interest  to  users,  the  price  a  manufacturer  charges  may  be  less  a 
function  of  the  computer's  technical  prowess  than  of  the  firm's 
marketing  strategy.  Therefore,  "cost"-based  technology  variables 
may  show  performance  correlations  due  more  to  market  forces  than 
to  an  underlying  SOA-based  relationship. 


In  passing,  Knight  provided  a  test  of  Grosch's  law.  The 
test  was  applied  over  two  sub-strata  of  the  sample,  and  concluded 
that  Grosch's  law  holds  for  the  computer  variables  he  studied; 


chat  is,  computing  power  increased  roughly  as  a  function  of  cost 
squared.  Where  Grosch's  law  would  have  predicted  an  "alpha 
parameter"  of  -0.5,  Knight  calculated  an  actual  value  of  -0.656 
for  the  1963-71  time  period  and  -0.668  for  1972-79.24 

Martino,  1985.  This  paper  by  Joseph  Martino  extended 
Dodson’s  earlier  work  on  ellipsoids  by  making  an  important 
change.  Martino  also  tested  the  model  on  four  data  sets.^  The 
alteration  he  made  was  to  allow  higher-order  versions  of  the 
surface  (where  an  ellipsoid  is  a  second-order  representation). 
His  data  sets  encompassed  clipper  ships,  jet  engines,  propeller- 
driven  aircraft  and  power  transistors. 

Allowing  higher-order  versions  of  the  surface  required 
modification  of  the  least  squares  fitting  procedure  used  by 
Dodson.  Rather  than  minimizing  the  mean  square  deviation, 
Martino  had  to  minimize  the  mean  absolute  deviation.  This 
required  that  the  surface  be  fit  by  iterative  experimentation 
and  inspection  rather  than  by  the  e xa ct -f it ti ng  algorithm 
developed  by  Dodson. 

Clipper  ships  were  divided  into  two  subsets  by  date  built, 
1820-1839  and  1840-1855.  (The  break  concurred  with  the 
development  of  a  design  method  which  used  iron  bars  to  minimize  a 
lengthy  ship's  susceptibility  to  a  problem  known  as  "hogging." 
This  development  allowed  designs  of  considerably  increased 
length,  thereby  increasing  performance.)  Three  variables  were 
used  to  describe  technology,  displacement,  length-to-camber  ratio 
and  draft.  The  surface  fit  was  of  order  six.  Results  were  as 


expected:  the  1840-1855  group  revealed  use  of  a  higher  level  of 
technology  than  did  the  1820-1839  group. 

Jet  engines  were  compared  using  thrust,  thrust-to-weight 
ratio  and  fuel  economy.  Second-order  surfaces  described  turbofan 
engine  technology  of  a  first  and  second  generation.  The  results 
were  generally  as  expected,  but  there  was  intersection  between 
the  two  surfaces  due  to  one  anomalous  engine.  A  surface  fit  to 
turbofan  versions  of  jet  engines  was  higher  than  a  surface  for 
contemporary  turbojets  with  no  overlap  between  the  groups. 

Propeller-driven  aircraft  were  compared  using  sixth-order 
surfaces  fit  to  structural  efficiency,  maximum  speed  and  useful 
load  through  four  generations  of  aircraft.  Surfaces  could  be 
fit  to  only  three  of  the  four  generations,  and  there  was  one 
intersection,  but  results  were  generally  good. 

Power  transistors  were  compared  using  four  variables;  power 
dissipation,  speed  of  operation,  maximum  allowable  voltage  and 
the  "percent  safe  operating  area."26  Surfaces  (of  unspecified 
degree)  were  fit  to  two  groups,  with  a  break  occurring  in  the 
late  1970s.  As  Martino  notes. 

The  tradeoff  surfaces  intersect,  which  means  that  the 
newer  technology  will  not  completely  displace  the 
older.  There  are  regions  of  the  parameter  space  where 
the  old  technology  is  still  superior  to  the  new.  .  .  . 
Nevertheless,  each  technology  does  in  fact  lie  on  a 
tradeoff  surface  which  defines  the  combinations  of 
parameters  accessible  to  the  designer. 2^ 
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From  the  Martino  piece  we  learn  that  the  ellipsoid  method, 
as  adapted,  works  well  with  a  wide  range  of  technologies.  It  is 
also  worth  noting  that  the  technologies  examined  encompass  at 
least  three  different  levels  of  system  aggregation.  The  clipper 
ships  and  propeller -driven  aircraft  are  whole  systems;  the  jet 
engines  are  sub-systems;  and  the  power  transistors  are  relatively 
minor  components. 

Literature  on  Cost  of  Advancing  SOA 

A  limited  amount  of  research  has  been  done  on  the  cost  of 
advancing  st at es -o f- th e- ar t .  To  date,  the  work  has  not  been 
definitive  in  terms  of  providing  answers,  but  it  does  provide 
insight  into  the  structure  and  difficulty  of  the  questiobs. 

Dodson ,  1977 .  The  focus  of  this  paper  by  Dodson  was  cost 

estimation — both  R&D  cost  and  procurement  cost. 28  The  empirical 
data  used  was  a  group  of  general  purpose  avionics  computers. 
Three  indicators  of  technological  capability  were  used, 29 

I ^  -  number  of  distinct  instruction  types  (add,  shift,  etc.) 

3 

X9  -  number  of  operations  per  second  (10  ) 

3 

■  density  of  the  central  processing  unit  (lbs/ft  ) 

A  multiple  linear  regression  was  run  where  the  dependent  variable 
was  the  year  of  development.  This  gave  the  following  equation, 

Ye  -  1961.3  +  0.03  lx  +  0.015  X2  +  0.06  X3 
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Dod  so  n 


Ye  was  calculated  for  each  computer  in  the  sample. 

reasoned  that  if  Y  _  .  <  Y  then  that  particular  computer  was 

developed  "ahead  of  it's  time" — that  the  SOA  had  been  advanced  by 

the  development  of  that  model.  Conversely,  if  Y  _  ,  >  Y  then 

r  1  actual  e 

the  computer  was  "behind  the  times".  Therefore,  Y  -  Y  ^  , 

r  e  actual 

measured  the  SOA  advance  for  each  computer. 

The  next  step  was  to  associate  SOA  advance  with  R&D  cost. 
This  was  done  with  another  regression.  The  result  was, 

Y  -  6.11  +  2.7  X1  -  4.57  X2  +  14.8  X3 

Where, 

Y  -  R&D  cost  (in  1974$106) 

X.  ■  SOA  advance,  Y  -  Y  .  . 

1  e  actual 

X2  ■  1  for  mic reprogrammable  computers,  0  for 
synchronous  computers 

X3  -  1  for  space  computers,  0  for  airborne  computers 

Dodson  reported  that  the  empirical  relationship  fit  to  this 
equation  was  "not  altogether  satisfactory."^  His  explanation 
for  the  lack  of  fit  was  that  there  are  other  cost  elements 
associated  with  R&D  which  were  not  identified  or  included  in  the 
study.  Therefore,  Dodson  attributes  the  inadequacy  of  the  model 
bo  a  lack  of  data  and  attendant  incomplete  modeling. 

Procurement  cost  was  estimated  from  a  cost  estimating 
relationship  (CER)  which  incorporated  the  year  of  technology  as 
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an  independent  variable.  According  to  Dodson,  this  is  necessary 
to  compensate  for  the  fact  that  production  technology  changes 

through  time.  A  more  advanced  design  would  be  more  costly  to 

produce  with  older  manufacturing  technology.  More  advanced 
production  technology  would  cause  the  procurement  cost  to  drop. 

Dodson  developed  the  following  regression  equation  from  data 
on  avionics  digital  processing  units, 31 

In  C  -  8.41  -  0.11  X.  +  0.249  X0  +  0.217  In  X_  +  0.274  In  X, 

1  1  3  4 

where, 

3 

C  *  cumulative  average  cost  at  Q  »  100  in  1974$10 

X^  ■  year  of  development  completion  -  1900 

X^  ■  0  if  microprogrammable ,  1  if  hard-wired  synchronous 

3 

»  (10  operations  per  second)(word  length,  bits) 

X^  »  number  of  distinct  instructions 

No  fit  statistics  are  reported  for  the  regression,  but  there  is  a 
plot  (shown  in  Figure  1.4  on  the  next  page)  which  seems  to 
indicate  Dodson  had  better  success  with  estimating  procurement 
costs  than  R4D  costs. 
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Figure  1.4  Dodson's  Computer  Production  Cost  Curve 

Dodson,  1985.  In  this  1965  article  Dodson  reports  much  of 
the  same  material  contained  in  his  1  977  piece,  but  with  some 
important  additions.^  Specifically,  he  introduced  the 
possibility  of  using  family  trees,  factor  analysis,  and 
technological  distance  scores. 

The  "family  trees"  concept  simply  structures  the  evolution 
of  successive  renditions  of  developing  technology.  For  example, 
Dodson  points  out  that, 

Individual  rocket  motors  have  "evolved"  directly  from 
some  predecessor.  .  .  .  each  successive  model  having 

one  or  more  refinements  or  changes  ...  It  then 
[becomes]  a  point  of  interest  to  develop  measures  of 
the  change  from  model  to  model,  our  hypothesis  being 
that  the  required  development  cost  would  be  positively 
related  to  the  amount  of  change. ^ 

The  simple  but  perhaps  often  overlooked  implication  is  that  R&D 
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cost  researchers  must  have  access  to  complete  generations  of 
technology  if  cost  is  to  be  properly  related  to  amounts  of 
change.  A  missing  generational  point  could  cause  the  amount  of 
change  effected  by  a  certain  expenditure  to  be  overstated. 

Factor  analysis  is  a  statistical  method  which,  today,  is 
more  commonly  used  by  researchers  when  investigating  social 
science  problems.  Basically,  it  accumulates  the  statistical 
influence  of  several  correlated  variables  to  form  "factors". 
The  method  frequently  enables  the  researcher  to  reduce  the 
information  content  of  a  large  number  of  variables  into  a 
relatively  small  number  of  factors  (or  c  om  po  site  variables). 
Factor  "scores",  the  calculated  values  of  the  composite 
variables,  can  then  be  used  for  further  analysis.  The  method  is 
of  considerable  advantage  when  sample  sizes  are  limited  because 
it  enables  subsequent  analyses  to  be  performed  with  more  degrees 
of  freedom.  An  analyst  can  also  use  axis  "rotation"  to  identify 
associative  relationships  where  none  were  apparent  from  the 
original  data. 3* 

Dodson  developed  nine  variables  describing  rocket  motor 
technology  and  evaluated  these  variables  across  a  sample  of  60 
motors  developed  between  1950  and  1968.  Then  he  ran  a  factor 
analysis  and  identified  four  factors  which,  collectively, 
explained  83. 5Z  of  the  variance  in  the  data. 3^  However,  the 
loadings  were  not  particularly  strong. 

Next  Dodson  described  "Technological  Distance  Scores". ^  A 
TDS  is  the  Euclidian  distance  between  two  systems,  where  the 


technology  location  of  each  system  is  measured  in  terms  of  N 
variables . 
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where , 


d^  .  -  Euclidian  distance  (or  technological  distance)  between 
J  the  ith  and  jth  systems 

Sik  ■  variable  value  of  the  ith  system  for  the  kth  variable 

■  variable  value  of  the  j_th  system  for  the  kth  variable 

N  -  number  of  variables 

Dodson  applied  the  TDS  measurements  to  his  sample  of  rocket 
motors  using  factor  scores  for  variables.  The  only  conclusion  he 
was  able  to  make  was  that  "TDSs  show  promise  of  being  a  useful 
indicator  of  development  cost  and  risk". 37 


Literature  on  Product  Cost  and  Advancing  SOA 


Alexander  and  Mitchell.  1985.  The  Alexander  and  Mitchell 
paper  addresses  product  cost  as  an  SOA  design  objective.  That 
is,  product  cost  is  viewed  as  one  among  many  performance 
variables  that  describe  the  technical  parameters  of  a  product. 3® 


The 

figure 

reproduced 

below  is  a 

summary  of  the 

concept 

.39 
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values 

of  other 
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variables  can 

always 
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achieved,  but  at  greater  cost.  To  move  from  curve  tQ  to  the  t^ 


Figure  1.5  Summary  of  Alexander  and  Mitchell's  Concept 

curve  (in  any  direction)  represents  an  SOA  advance.  The  vertical 
bell-shaped  curves  portray  cost  uncertainty  for  given  levels  of 
performance . 

Alexander  and  Mitchell  examined  several  sets  of  data 
including  milling  machines  from  the  early  1860s  through  the 
1970s,  and  commercial  turbine-powered  aircraft  from  1958  through 
the  early  1980s. Their  results  show  inflation-adjusted  product 
costs  ri si ng  over  time  after  adjusting  for  level  of  product 
technology.  The  following  is  a  result  for  the  cost  of 
airframes:^* 

ln(airframe  cost)  -  -9.13  +  0.91  ln(airframe  weight) 

♦  0.0039  time 

R2  -  0.93,  SEE  -  0.19,  N  -24 
where , 

airframe  cost  -  cost  of  aircraft  minus  engines,  deflated  by 
airframe  cost  index 
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airfraae  weight  ■  weight  of  airframe  (aircraft  empty  weight 
minus  engine  weight) 

time  »  aircraft  certification  date;  quarter  years  since 
1942,  third  quarter. 

However,  it  is  crucial  to  note  that  the  "cost"  referred  to 
here  is  the  purchase  price  of  the  unit  of  equipment  itself.  The 
total  cost  of  using  the  unit  for  its  intended  purpose  is  an 
entirely  different  matter.  (The  two  concepts  are  analogous  to 
the  DoD  notions  of  procurement  cost  versus  life-cycle  cost.) 
Alexander  and  Mitchell  point  out  that, ^2 

Productivity  estimates  based  on  product  characteristics 
may  not  adequately  reflect  the  total  value  of  the 
product  to  users,  partly  because  of  the  noninclusion  of 
all  the  relevant  characteristics.  For  example,  seat- 
mile  costs  are  a  function  of  the  number  of  seats, 
operating  costs,  fuel  costs,  and  speed  as  well  as  of 
aircraft  price. 

The  graph  below  (See  Figure  1.6  on  the  next  page.)  clearly 
indicates  that  the  cost  per  seat-mile  for  commercial  aircraft  has 
dropped  significantly  over  the  last  twenty  years.  Even  if  other 
performance  characteristics  had  remained  constant  this  would 
represent  a  significant  technology  advance  in  terms  of  utility  to 
the  user. 

The  most  important  concept  to  emerge  from  the  Alexander  and 
Mitchell  work  is  that  product  cost  should  not  necessarily  be  a 
design  variable;  the  more  appropriate  measure  of  economy  is  life- 
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cycle  cost.  But  even  then,  there  are  other  factors  which  may 

override  cost. ^3 

Noise,  safety,  pollution,  comfort,  convenience, 
flexibility,  range,  reliability,  and  countless  other 
attributes  are  routinely  considered  by  aircraft  buyers 
and  their  customers. 


and  development  managemen t . 44  Among  the  topics  treated  were, 
Methodology  for  integrated-systeo  R  and  D,  system 
design  concepts,  major  system  components,  matrix 
organization,  cost  effectiveness,  life-cycle  cost,  and 
customer -acceptance  requirements. 

Two  of  the  more  useful  concepts  Hovanessian  identified  were 
an  expanded  set  of  user-oriented  product  characteristics  and  a 
comprehensive  illustration  of  life-cycle  cost. 

Hovanessian,  like  Alexander  and  Mitchell,  points  out  that 
user  utility  of  a  technological  system  is  only  partially 
described  by  using  scientific  variables  alone.  Equally  important 
in  his  mind  are  "customer  acceptance"  parameters  "which  include 
such  factors  as  maintainability,  availability,  degree  of 
automation,  operator  approval,  and  improvements  over  previous 
systems . "^6 

Ho  vane ss ian  '  s  representation  of  life-cycle  cost  includes 
three  phases;  RDTAE ,  investment  and  operating.  He  describes 
these  phases  in  terms  of  rates  of  spending  as  a  function  of  time. 
Each  phase  can  therefore  be  represented  as  a  spending  pattern. 

While  Ho  vanessian  ’  s  spending  patterns  are  not  empirically 
based,  it  is  easy  to  see  that  empiricism  could  be  useful  to  the 
proposal  evaluation  process  in  at  least  two  ways.  First,  the 
present  value  of  the  cost  of  alternate  technologies  could  be 
compared  with  each  other  and  with  other,  more  technical, 
descriptions  of  the  proposals.  Second,  budgeted  to-date  and 
actual  to-date  spending  could  be  compared  for  control  purposes. 


Lieahard.  1979.  This  paper  studies  the  rate  at  which 
technology  is  improved,  and  how  (whether)  this  rate  has  changed 
through  time.^  The  basic  metric  Lienhard  uses  is  the  order  of 
increase,  or  "n_-f  olding" ,  that  has  taken  place  during  a  working 
"lifetime",  which  he  defines  as  30  years.  The  formulation  he 
uses  is, ^8 

—  -  n(t  “  t0)/3°  .  e(t  '  t0)C(ln  n>/3°l 

% 

where , 

Q  ■  the  value  of  the  attribute  to  be  improved  measured  at 
time  t 

Qq  ■  the  initial  value  of  the  attribute  quantity,  measured 
at  time  tg 

Lienhard  points  out  that  an  alternate  way  to  look  at  this  formula 
is  to  think  of  30/ln(n_)  as  the  "time  constant,"  T--the  time 
required  to  complete  one  e-folding  of  Q. 

It  should  be  noted  that  Lienhard's  work  is  based  on  single¬ 
variable  measurements.  In  other  words,  only  one  attribute  of  the 
technology  in  question  is  tracked  through  time. 

Several  forms  of  technology  were  studied.  The  results  can 
be  summarized  in  a  single  table:^9 

Technology  Attribute  Dates  n^ 

Mechanical  clock  Accuracy  1400-1920  1.95 

Steam  power  Thermal  efficiency  1742-1850  2.5 

Land  transport  Speed  1803-1965  1.86 

Low  temperatures  Difference  in  temper  1860-1936  21.5 
Air  transport  Speed  1884-1967  10.1 


Lienhard  makes  several  observations  based  on  his  analysis. 
First,  he  states  that  technology  advance  sometimes  depends  on 
motivation.  For  example,  regarding  air  transport  speeds  he 
states  that,-^ 

It  seems  unlikely  that  there  remains  much  motivation  to 
continue  increasing  air  speeds.  They  have  now  reached 
such  enormous  values  that  problems  of  starting, 
stopping,  and  turning  determine  how  fast  a  person  can 
be  taken  from  one  place  to  another. 

Another  interesting  observation  is  that  one  technology  must 
sometimes  "wait"  on  another.  For  example,  Rozier  experimented 
with  balloon  flight,  reaching  a  maximum  height  of  about  9,000 
feet  in  1783.  Then  little  progress  was  made  for  about  a  century 
and  a  half  while  aviation  awaited  the  development  of  high- 
altitude  breathing  equipment.  Finally,  rapid  advance  was  again 
made  from  1923  until  1957,  "when  orbital  vehicles  constituted  a 
replacement  technology . * 

Lienhard  also  observed  that  the  rate  of  change  in 
technological  growth  increased  dramatically  in  the  middle  of  the 
19th  century.  Other  authors  have  observed  this  as  well. 
Lienhard's  explanation  is  that,  "Suddenly,  technology  began  to 
breed  more  technology."^ 

His  final  observation  is  potentially  the  most  useful.  He 

say  s , 

[An]  empirical  fact  which  we  have  heretofore  taken  for 
granted,  but  which  the  data  show  to  be  unwaveringly 
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true,  is  that  the  rate  of  improvement  of  a  particular 

technology,  once  established,  does  not  change. 53 
If  this  is  literally  correct  (and  his  data  do  tend  to  support  the 
observation),  there  are  some  major  implications  for  the  cost,  and 
even  the  feasibility,  of  undertaking  advances  in  SOA. 
Specifically,  the  anticipated  advance  could  be  expected  to  occur 
only  by  some  quasi-naturally  established  date.  Attempts  to 
accelerate  this  natural  process  would  undoubtedly  be  very  costly. 
It  would  follow  that  those  who  express  technology  levels  as 
simply  a  "date”,  or  as  a  "year  of  technology",  have  good  reason 
for  doing  so ! 

Foster,  1986.  Technology  advancement  forecasting  is  really 
the  controlling  theme  of  this  1986  paper  by  Foster. 54  He  opened 
with  a  familiar  but  important  discussion  on  the  identification  of 
performance  parameters.  He  pointed  out  that,  "The  performance 
parameters  .  .  .  must  be  related  to  key  design  factors",  and 
that,  "The  best  approach  is  to  pick  a  few  areas  that  are  critical 
and  concentrate  on  them."  He  also  added  that  it  is  "important  to 
recognize  that  the  performance  parameters  will  change  over 
time . "55 

Foster's  discussion  then  turned  to  forecasting  the  future  of 
technology  by  fitting  S-curves  and  determining  limits.  He  does 
not  offer  very  specific  solutions,  but  he  does  articulate  the 
possibilities  of  physical  limits,  and  he  identifies  questions 
that  relate  to  mechanisms  that  might  limit  the  performance  of 
the  relevant  technology:  "Will  it  be  thermodynamics?  Strength  of 
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materials?  Chemistry?  Laws  of  motion?  Some  fundamental 
physical  force?"56 

Foster  then  turned  to  S-curve  fitting.  Here  he  pointed  out 
the  importance  of  historical  analysis  and  suggested  the  use  of 
"logistics"  or  "Gompertz"  curves.  Finally,  he  warned  against 
over -simplicity: 

The  simplest  approach  is  to  make  the  top  part  of  the 
curve  symmetrical  with  the  bottom  part  and  draw  a 
straight  line  connecting  the  points.  This  approach  is 
fast  and  cheap.  It  also  produces  the  least  insight. 

One  very  valuable  insight  Foster  provided  was  to  synthesize 
earlier  work  by  Putnam  on  the  relationships  between  development 
cost  and  development  time. 57  Foster's  formulation  was, 58 

a  b 

Project  cost  •  Projected  performance  /[Efficiency  x  Time  ] 

where  a_  and  b^  are  parameters.  Foster  suggests  the  parameters  may 
be  specific  to  individual  contractors.  Note  that  (if  b_  is 
positive)  a  reduction  of  the  time  available  for  the  undertaking 
tends  to  increase  project  cost. 

Becker  and  Speltz,  1986.  This  paper  reports  another  effort 
to  plot  S-curves  and,  in  particular,  to  forecast  the  "turning 
point",  or  the  point  at  which  the  advance  of  the  subject 
technology  will  cease. 59  The  specific  technology  studied  was  a 
class  of  insecticides  known  as  pyrethroids.  "The  question  facing 
the  RAD  management  was  whether  or  not  research  in  the  pyrethroids 
should  be  continued."^ 


The  procedure  was  reported  only  in  broad-brush  terms,  and 
apparently  was  accomplished  principally  from  visual  inspection  of 


historical  plots.  The 
work  on  organophosphate 


illustration  below 
insecticides . ^1 
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Figure  1  —The  S-curve  of  Technical  Progress 
Versus  Effort 
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Figure  j  —The  \-Curre  for  Figure  v  — Predicting  the  Shape  of  the  5- 

f irganophosphate  Insecticides  Curve 
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The  curve  in  Becker  and  Speltz  '  Figure  1  shows  the 
theoretical  advance  of  technology,  while  Figure  2  demonstrates 
marginal  payoff  for  incremental  research  effort  (productivity). 
Becker  and  Speltz  suggest  that  research  efforts  should  be 
abandoned  once  marginal  payoff  passes  the  peak,  or  just  past  the 
inflection  point  in  Figure  1.  (This  is  a  premise  with  which 
economic  theorists  would  disagree.  The  more  generally 
recommended  position  would  be  that  effort  should  continue  so  long 
as  marginal  benefits  exceed  marginal  costs.) 

Figure  3  above  is  an  attempt  to  plot  an  S-curve  for 
organophosphate  insecticides.  The  plot  implies  that  the 
underlying  technology  had  matured  by  about  .19  7  4  or  1975  . 
However,  it  is  worth  noting  that  an  analyst  positioned  in,  say, 
1955  or  in  1962  might  easily  have  drawn  the  incorrect  conclusion 
that  the  curve  had  peaked.  Ex  ante  estimates  of  the 
technological  limits  are  at  best  difficult  to  make.  The 
projection  labeled  "A"  in  Figure  4  illustrates  the  magnitude  of 
error  that  might  easily  be  committed. 

Becker  and  Speltz  completed  their  analysis  by  examining  the 
state  of  development  of  five  "distinguishable  attributes".  The 
attributes  were. 

o  Spectrum  of  activity 

o  Increased  soil  activity 

o  Systemic  activity 

o  Reduced  fish  toxicity 

o  Reduced  cost 
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Their  conclusion  was  chat  improvements  which  would  represent  a 
technological  advance  for  the  pyrethroids  were  not  likely." 

CONCLUDING  COMMENTS 

This  chapter  began  by  pointing  out  that  the  cost  of 
developing  S  OA  extensions  relates  to  the  scale  of  the 
undertaking.  Then  the  more  important  literature  in  the  field 
was  surveyed  in  order  to  examine  the  techniques  currently 
available  for  costing  development  contracts.  In  this  concluding 
section  the  objective  will  be  to  step  back  and  examine  SOA 
costing  trends  and  the  challenges  that  lie  ahead. 

SOA  Measurement 

The  art  of  measuring  SOA  extensions  was  first  approached 
with  rigor  by  Dodson  and  Graver,  who  reported  an  important 
theoretical  advance  in  1969. 62  The  innovation  represented  by 
their  work  was  the  use  of  (convex)  ellipsoidal  hypersurfaces  to 
portray  stages  of  SOA.  The  level  of  SOA  advance  incorporated  in 
a  system  was  represented  by  the  radial  distance  of  a  desired 
point  from  the  most  current  ellipsoid. 

In  order  to  make  use  of  the  Dodson  and  Graver  approach 
technological  parameters  must  be  specified  in  such  a  way  as  to 
collectively  describe  the  overall  level  of  technology  embodied  in 
the  systems  or  subsystems  studied.  Dodson  and  Graver  took  care 


to  point  out  that  if  the  technique  was  to  be  useful,  the 
parameters  used  "should  be  among  those  that  are  typically 
ascertainable  during  the  early  decision-making  stages  of  the 
system  life-cycle. "63 

Alexander  and  Nelson's  1972  RaND  report  articulated  an 
important  limitation  of  the  Dodson  and  Graver  approach — that  is, 
the  progression  of  technology  must  conform  to  an  assumption  of 
"continuity 64  In  effect,  this  limitation  qualifies  parametric 
SOA  measurement  models  for  use  only  when  measuring  evolutions  in 
technology . 

Alexander  and  Nelson,  like  Dodson  in  1977,65  analyzed 
empirical  data  to  test  SOA  measurement  theory,  but  both  they  and 
Dodson  were  able  to  use  only  linear  methods,  not  ellipsoids,  with 
any  degree  of  success.  Their  model  used  the  "date  of  technology" 
as  the  dependent  variable.  A  45*  line  thus  defined  an  expected 
date  of  technology  over  the  period. 

Others  have  tried  second-order  versions  of  SOA  equations, 
and  even  multiplicative  models.  The  1981  work  of  Gordon  and 
Munson  is  an  example.  They  added  insight,  but  at  the  empirical 
testing  stage  of  their  analysis  they  had  to  resort  to  subjective 
methods  to  establish  weights  for  parameter  influence. **6 

Martino  extended  Dodson’s  work  by  allowing  higher-order 
equations,  but  he  was  forced  to  use  iterative  experimentation  and 
inspection  as  a  fitting  method.*’7  On  the  other  hand,  Knight  used 
Grosch's  law  and  empiricism  to  argue  for  the  fundamental 
correctness  of  second-order  relationships.  Grosch's  law,  "would 


have  predicted  an  "alpha  parameter"  of  -0.5;"  Knight  calculated 
an  actual  value  of  -0.656  for  the  1963-71  time  period  and  -0.668 
for  1972-79. 68 

SOA  Extension  Costing 

There  have  been  almost  no  really  successful  studies  relating 
development  cost  with  SOA  extension.  Perhaps  the  only  one  of 
any  significance  was  published  by  Dodson  in  1977. 

Dodson  attempted  to  establish  a  relationship  between  degrees 
of  SOA  extension  and  development  cost  by  first  running  a  linear 
regression  in  which  various  technology  parameters  were  the 
independent  variables  and  the  dependent  variable  was  the  year  of 
development.  69  Residuals  from  this  regression  then  became  an 
independent  variable  which  was  regressed  against  development 
cost.  Dodson  reported  that  the  fit  of  this  equation  was  "not 
altogether  satisfactory."^ 

Suggestions  for  Future  Improvements 

This  literature  review  suggests  several  ways  in  which  our 
ability  to  measure  and  cost  SOA  extensions  may  be  improved. 
First,  more  accurate  SOA  measurements  should  be  developed.  This 
effort  will  call  for  empirically  based  parametric  models  with 
somewhat  more  detailed  input. 

Gordon  and  Munson  (1981)  were  the  first  to  suggest  the  use 
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of  factor  analysis  as  a  method  for  grouping  proposed  technology 
parameters  into  clusters  that  have  similar  behavior  and 
influence  on  SOA.^l  The  advantage  of  this  method  is  that  the 
effects  of  many  variables  can  be  condensed  into  a  manageable 
number.  But  the  method  has  not  been  tested  in  terms  of  its 
promise  for  improving  cost  estimation  accuracy. 

The  technologies  that  have  been  examined  in  the  SOA 
measurement  work  encompass  at  least  three  different  levels  of 
system  aggregation — all  relatively  successfully.  This  implies 
chat  appropriate  data  bases  might  be  enlarged  by  examining 
technologies  at  the  subsystem  level  when  one  subsystem  is  used 
in  more  than  one  system. 

A  additional  untested  suggestion  is  that  "technological 
distance  scores"  (determined  from  ellipsoids)  might  be  a  useful 
measure  of  SOA  extensions . ^2  However,  Dodson's  attempt  to  use 
them  was  unproductive;  the  only  conclusion  he  was  able  to  make 
was  that  use  of  technological  distance  score  measures  "shows 
promise". 

Lienhard's  1  979  study  of  the  rate  at  which  technology  is 
improved  provided  a  provocative  suggestion  that  needs 
consideration.^  He  suggested  there  is  an  essentially  fixed  rate 
at  which  any  given  technology  can  be  improved. 

[An]  empirical  fact  which  we  have  heretofore  taken  for 
granted,  but  which  the  data  show  to  be  unwaveringly 
true,  is  that  the  rate  of  improvement  of  a  particular 
technology,  once  established,  does  not  change. ^ 


If  indeed  this  is  true,  there  are  major  implications  for  the 
cost,  and  even  the  feasibility,  of  attempting  to  undertake 
advances  in  SOA  in  a  more  compressed  time  frame. 

Alexander  and  Mitchell  argue  that  product  cost  should  not 
necessarily  be  a  design  variable;  a  more  appropriate  measure  of 
economy  is  life-cycle  cost. 75  Ho vanessian '  s  representation  of 
life-cycle  cost  includes  three  phases;  RDT&E,  investment  and 
operating. 76  jj0  methods  for  incorporating  life-cycle  cost  have 
yet  been  developed. 

Cost  elements  associated  with  R&D  which  have  not  been 
identified  or  included  in  a  study  can  lead  to  incomplete 
modeling.  The  challenge  is  to  more  completely  identify  cost 
variables.  Dodson  argues  that  R&D  cost  researchers  must  have 
access  to  complete  generations  of  technology . 77  Otherwise,  the 
continuity,  and  value,  of  a  data  base  is  severely  weakened. 

Finally,  the  "S"  shape  of  the  SOA  progression  curve  has  also 
been  examined.  Foster's  discussion  addressed  forecasting  the 
future  of  technology  by  fitting  S-curves  and  determining 
limits. 78  But  the  Becker  and  Speltz  piece  showed  the  great 
magnitude  of  the  error  that  might  be  committed  given  the  present 
state  of  knowledge  —  particularly  when  turning  points  are 


forecasted  ex  ante. 
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Chapter  2 


MEASURING  TECHNOLOGY 

INTRODUCTION 


The  objective  of  this  chapter  is  to  formulate  a  measure  of 
overall  technology  that  is  demonstrably  applicable  to  the  systems 
studied,  and  that  will  serve  as  satisfactory  input  for  the  next 
step,  the  cost  estimation  process,  to  be  examined  in  Chapter  3. 
The  working  assumptions  used  in  this  chapter  are: 

1.  The  ellipsoid  method  developed  by  Dodson  and  Graver  is 
theoretically  correct  in  its  second-order  form.l 

2.  Factor  analysis  is  an  appropriate  methodology  for 
capturing  the  influence  of  a  relatively  large  number  of 
technological  parameters. 2 

3.  Any  data  set  characterized  by  "technological 
continuity"  can  be  used  for  empirical  demonstration  of  the 
methodology  to  be  developed.  3 

DATA  SET 

Obtaining  reliable  data  in  sufficient  quantity  to  allow 
rigorous  inquiry  is  a  constant  problem  for  cost  analysts.  Dodson 
has  pointedly  illustrated  the  usual  scope  of  this  problem  in 
Figure  2 . 1  below. ^ 


Figure  2.1  Dodson's  View  of  Data  Collection 

Data  describing  various  technological  and  cost  parameters 
for  18  satellite  programs  were  supplied  by  the  United  States  Air 
Force. ^  The  satellites  described  included  several  types, 
communications,  surveillance,  navigation,  etc,;  they  were  first 
launched  between  1966  and  1986. 

Variable  Selection 


The  first  step  was  to  select  variables  that  describe  the 
technology  embedded  in  satellites.  Data  describing  85  technical 
characteristics  of  each  of  the  18  satellites  were  available.  The 
85  properties  are  listed  on  the  next  page. 6 
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satellite  data 


ACS:  SLEW  RATE 
ACS:  ORIENTATION 
ACS:  POINTING  ACCURACY 
ACS:  JIT  REQ 
ACS:  JIT  AMP 
ACS:  JIT  FREQ 
ACS:  PROPELLANT  WT 
ACS:  TOT  IMPULSE 
ACS:  RCS  FUNCTION 
ACS:  PRIM  STAB  METHOD 
ACS:  TORQ  METH-COLD  GAS 
ACS:  TORO  METH-MONOP 
ACS:  TORQ  METH-BIPROP 
ACS:  TORQ  METH-MACN 
ACS:  TORQ  METH-WHEELS 
ACS:  TORQ  METH-CMG 
ACS:  MOM  MCMT -COLD  CAS 
ACS:  MOM  MCMT-MONOP 
ACS:  MOM  MGMT-BIPROP 
ACS:  MOM  MCMT-MAC 
ACS:  MOM  MCMT-SOLAR 
ACS:  MANEUVERABILITY 
ACS:  STATION! EEPINC 
ACS:  PER/PHASE  CONTROL 

LAUNCH  METHOD 
ORBITAL  APOGEE 
ORBITAL  INCLINATION 
DESIGN  LIFE 
POINTING  ACCURACT 
NUCLEAR  HARDENING 
NUMBER  OF  appendages 
QUALITY:  ZCLASS  B 
QUALITY:  ZCLASS  S 

STRUC :  ZDEPLOYED  WT 
STRUC :  MAT-ALUM  IN 
STRUC:  MAT-MACNES 
SI  RUG:  MAT-SIMP  ALLOTS 
STRUC:  MAT-FIBERGLASS 
STRUC:  MAT-TITANIUM 
STRUC:  MAT-HONEYCOMB 
STRUC:  MAT-BLRYLLI UM 
STUCT:  MAT-BORON/GRAPH 
STRUC:  MAT-HYBRID/MET  MAT 
STRUC:  TYPE  CONSTRUCTION 

COMM:  NO.  CHANNELS 
COMM:  NO.  TRANSMITTERS 
COMM:  NO.  RECEIVERS 
COMM:  ANIT-JAK  CAPABILITY 
COMM:  POWER  REQ(W) 


AIM:  THRUST  VFCT  CONT 
AIM :  DfLTA  V 
ARM:  NOZZLE  MATERIAL 
ARM:  CASE  COMP 
ARM:  DRY  WEIGHT 
ARM:  PROP  WEIGH! 

ARM:  SPEC  IMPULSE 
ARM:  BUPN  TIME 
ARM:  TOTAL  IMPULSE 
ARM:  PAYLOAD  WEIGHT 

EPS:  ARRAY  TOPOLOGY 
EPS:  STORACE  TYPE 
EPS:  BOL  POWER  (W) 

FPS:  ARRAY  AREA( SQ  FT) 
EPS:  TOTAL  NO.  CELLS 
EPS:  CFLL  EFFICIENCY 
EPS:  BAT  CAP  (AMP-HRS) 
EPS:  NO.  BATTERIES 
EPS:  CELLS  PER  BATTERY 

1HERM:  PASS-COATINCS 
THERM:  PASS-SLI 
THERM:  PASS-MLI 
THERM:  PASS  CONDUCTION 
THERM:  PASS-CCHP 
THERM:  ACT-VCHP 
THERM:  ACT-1  PH  f  L  LOOP 
THERM:  ACT-CAPPIL  FL  LP 
THERM:  ACT-Z  PH  FL  LOOP 
THERM:  HEAT  REJ  TYPE 
THERM:  RADIATOR  ORIENT 
THERM:  MAI  TEMP(C) 

THERM:  MIN  TEMP(C) 

THERM:  DUTY  CYCLF(Z) 
THERM:  DES  COMPUTER  HRS 

TTC:  NO.  CHANNELS 
TTC :  NO.  TRANSMITTERS 
TTC:  NO.  RECEIVERS 
TTC:  ANTI-JAM  CAP 
TTU:  1  APE  R-BIT  CAP 
TTC:  TAPE  R-RATE(RBPS) 
TTC:  TAPE  R-REC  TIME(M;N) 
TTC:  POWER  REQ(W) 

TTC:  TAPE  R-QUANTITY 
TTC:  TAPE  R-POWER  REQ(W) 
TTC:  AUTON  OPS(DAIS) 

TTC:  NO.  COMMANDS 


Figure  2.2  85  Properties  Describing  Satellite  Technology 


Some  of  the  85  properties  included  in  the  data  set  can  be 
considered  design  objectives,  but  many  others  are  simply 
byproducts  of  the  design,  or  are  not  stated  in  a  form  that  really 
says  a  great  deal  about  the  technology  embodied.  Therefore, 
technical  expertise  was  essential  in  identifying  and,  in  some 
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cases  reconstructing,  relevant  variables. 

Extensive  data  review,  conversations  and  conferences  took 
place  among  a  group  of  satellite  experts  at  the  Naval 
Postgraduate  School: 

Dr.  Allen  E.  Fuhs,  Code  72 

Distinguished  Professor  of  Aeronautics  &  Space 
Dr.  Richard  V.  Adler,  Code  62Ab 

Adjunct  Professor,  Elec.  &  Comp.  Engineering 
Mr.  Marty  Mosier,  Code  72 

Staff  Engineer,  Space  Systems 

This  effort  led  first  to  the  construction  of  several  composite 
variables  based  on  the  guideline  recommendations  originally 
suggested  by  Dodson  and  Graver  in  1969: 

First , 

The  selection  of  SOA-determining  characteristics  should 
be  concentrated  on  those  that  are  at  least  partially 
influenced  by  engineering  development  decisions.  The 
individual  characteristics  .  .  .  represent  constraints 
upon  the  achievement  of  other  design  characteristics 
which  are — to  a  degree — goals  of  design. ^ 

Second , 

.  .  .  Characteristics  should  be  specified  so  that 

increasing  values  of  the  characteristics  correspond  to 
greater  technical  difficulty.8 


Finally  , 


.  .  .  The  characteristics  should  be  among  those  that 

are  typically  ascertainable  during  the  early  decision¬ 
making  stages  of  the  system  life-cycle. ^ 

The  last  of  these  guidelines  is  particularly  important  if  the 
final  analytical  output  is  to  be  useful  for  early  feasibility 
study,  or  when  development  funding  is  being  sought,  or  when 
attendant  budget  decisions  are  being  made. 

The  result  of  the  conferences  was  consensus  identification 
of  18  variables  or  composite  variables  that  can  be  used  to 
describe  satellite  technology: 

Attitude  Control  System  (ACS)  variables: 

ACS1  —  Reciprocal  of  Pointing  accuracy. 

ACS2  —  Primary  stabilization  method  —  coded; 
spin  -  0,  moment,  inertial,  dual  -  1. 

ACS3  ' —  Maneuverability,  yes-no  —  coded; 
no  ■  0,  yes  -  1. 

Apogee  Kick  Motor  (AKM)  variables: 

AKM1  —  Specific  impulse. 

AKM2  —  Propellent  weight  /  Dry  weight. 

Communications  variable: 

COMM  --  Power  required. 

Electrical  power  systems  variables: 

EPS1  —  Battery  Capacity. 


EPS2  —  Beginning  of  life  power  /  Array  Area, 
compensated  for  stabilization  &  array  deployment. 
EPS3  —  Array  topology  —  coded; 

Body  ■  0,  Fixed  ■  1,  Movable  ■  2 


Mission  or  environmental  variables: 

LIFE  —  Design  life. 

NHARD  —  Nuclear  hardening  —  coded; 

Geo  ■  Leo  »  4,  Elliptical  ■  7,  War  •  10 
LAUNCH  —  Launch  method  —  coded; 

ELV  *  0,  Shuttle,  Dual  -  1 
QUALS  —  Quality  percent  class  S 

APOGEE  —  Orbital  apogee  *  Design  life  *  (Z  Quality  S  + 
0.8  Z  Quality  B)  /  10000 

DESIGN  —  Design  life  /  (Z  Quality  S  +  .1  Z  Quality  B). 


Structure  variable: 


STRUC  —  Percent  deployed  weight. 


Thermal  variable: 

THERM  —  Max  temperature  -  Min  temperature. 


Tracking  telemetry  control  variable: 

TTC  —  Autonomous  operating  days. 

These  18  variables  were  determinable  for  each  of  18 
satellites,  with  one  exception;  THERM  was  missing  for 
Satellite  R.^  The  variables  and  their  values  are  listed  on  the 
next  page. 
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Figure  2.3  18  Variable  Values  for  Each  Satellite 


FACTOR  ANALYSIS 


The  values  of  all  18  variables  for  all  18  satellites  were 
loaded  into  an  SPSS*  (Release  2.1)  data  file.  A  factor  analysis 
was  run,  asking  first  for  the  correlation  matrix  and  1-tailed 
significance  levels.  (The  output  is  shown  on  pages  53  and  54.) 
The  data  are  quite  robust  with  high  coefficients  of  variation  and 
many  significant  correlations. 

Since  there  are  too  many  variables  (18),  the  matrix  was 
ill-conditioned.  Some  variables  had  to  be  eliminated  from  the 
analysis.  Accordingly,  variables  were  given  an  additional 
screening.  Rejection  would  have  been  based  first  on  n_o 
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correlations  with  .05  or  better  significance.  However,  no 
variables  could  be  eliminated  on  this  basis. 

STRUC  was  eliminated  because  the  engineers  felt  some  of  data 
were  incorrect.  However,  the  remaining  17  variables  were  all 
candidates  for  further  consideration. 

The  variables  were  next  subjected  to  a  principal  components 
factor  analysis  using  the  varimax  procedure  for  orthogonal 
rotation.  Variables  with  large  (greater  than  0.50)  negative 
loadings  were  eliminated  based  on  Dodson’s  second  guideline. 
This  eliminated  DESIGN,  ACS3  and  EPS2.  QUALS  was  discarded  next, 
because  it  had  no  substantial  factor  loading  (no  loading  equal  to 


or  greater  than  0.50). 
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The  remaining  13  variables  were 


factored  again  and  clustered  very  nicely  onto  four  factors  with 
78. 5%  of  the  variance  explained. 
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FACTOR  ANALYSIS 


ANALYSIS  NUMBER  l  PAIRWISE  DELATION  OF  CASES  WITH  MISSING  VALUES 


«AN 

STQ  OEV 

CASES 

LABEL 

ACS1 

3.70  722 

3.71033 

It 

*e: IP (POINTING  ACCURACY) 

ACS* 

.77779 

.42771 

18 

PRIM  STAB  METHOD—  S«0.  M.J.D-1 

ACSJ 

03333 

.30340 

10 

MANEUVERABILITY— N-0.  Y-l 

AKMl 

145.74144 

147.15211 

18 

SPEC  IMPULSE 

AKM2 

54.033 

5.77103 

10 

PRO F  WEIGHT/ORY  WEIGHT 

CCMN 

275- 32444 

310.02711 

18 

POWER  RCOCWJ 

E ps: 

4030001 

113. .502 1 

10 

BAT  CAP  CAMP -MRS  ) 

EPS2 

15.4455* 

14.11114 

18 

(BOL  POWER (W)/ ARRAY  AREA (SO  FT) )»PI-N 

eps: 

.00001 

.10025 

10 

ARRAY  TOPOLOGY—  B«0.  F»l.  M»2* 

LIFE 

S3««*«7 

50.05411 

;o 

DESIGN  LIFE 

NHARO 

$.33333 

2. 11430 

10 

NUCLEAR  HARDENING — G.L.4 ,  £>7.  W«10 

launch 

.4 2771 

10 

LAUNCH  method— clv.o.  S.D*l 

slals 

27.7*770 

42.77120 

10 

QUALITY  xclass  s 

APOGEE 

70  U.  12270 

7105.15315 

10 

orbital  APOGEC-OESIGN  LIFE* (EQUALITY'S* 

DESIGN 

2.05590 

2.00150 

10 

DESIGN  LIFE/ (EQUAL I TVS  •  . 1EQUALITYB) 

3TRUC 

0.57001 

7.10019 

10 

E DEPLOYED  WT 

THERM 

21.27770 

14.43042 

10 

MAX  temp-min  TEMF 

TTC 

0.02770 

21.02011 

19 

AUTON  ops (Days ) 

CORRELATION  MATRIX* 


ACS  1 

ACS2 

ACS3 

AKMl 

AKM2 

COMM 

EFSl 

EFS2 

EFS3 

LIFE 

MHARO 

ACS1 

1.00000 

ACS2 

. 5B550 

1.00000 

AC  S3 

.21354 

.47001 

1.00000 

AKMl 

-.30217 

-.20045 

-.15520 

1.00000 

AX  M2 

-.24407 

-.14203 

-.12000 

.17472 

1.00000 

CCHM 

.44111 

.37127 

.30074 

.00454 

.10134 

1.00000 

EPS  I 

-.00570 

.2517* 

.25112 

-.2*221 

-.2235* 

.02012 

1.00000 

EPS2 

-.25000 

-.05074 

-.4405* 

.40105 

.30200 

-.20714 

.12120 

l.  90000 

EPS5 

.01704 

.54500 

.20510 

-.12054 

.01030 

.55171 

.291*1 

-.27154 

1.00000 

LIFE 

.4*405 

.  1525* 

.41012 

.01275 

. 12223 

.724*3 

-.04157 

-.20*22 

.55177 

l. 00000 

W4AR0 

.  14570 

-.0455* 

.07255 

.1504* 

.204*7 

.20914 

-.04055 

-.13200 

.2*703 

.15*3* 

1.00000 

LAUNCH 

.40170 

.20571 

.25105 

-.002*7 

.12152 

.04244 

.13*05 

-.2753* 

.*7004 

.*002* 

.43555 

OUALS 

.50122 

.05571 

.21001 

.15450 

.10211 

.41700 

.00570 

-.552*2 

.31054 

.55515 

.54*04 

APOGEE 

.42054 

.04325 

.57*54 

.11*25 

.1504  3 

.74*12 

-.05071 

-.21*77 

.25530 

.17407 

.11047 

DESIGN 

-.04071 

.131*1 

.17105 

. 111*7 

.20500 

.11750 

-.20104 

.1574* 

-.10*00 

.37012 

-.34005 

STRUC 

.2*105 

.42244 

.  30244 

-.5255* 

-.21430 

.37000 

.7121* 

-.1477* 

.*1752 

.02*34 

.014** 

THERM 

"4002 

-.20571 

-.522*1 

.03212 

.05043 

.IU41 

-.01350 

-.04055 

.  17413 

.10340 

.31540 

TTC 

.552*1 

.01342 

.17001 

-.11771 

-.21104 

.14417 

.05020 

-.17400 

.21013 

.52257 

.4*415 

Figure  2.4  Initial  Factor  Analysis 


LAUNCH 


1.00000 
.2997 X 

.10425 

. 20591 

.mu 
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P*CTO 

R  A  N  A  L  V  S  I 

S  -  -  -  - 

CUALS 

APOGEE  DESIGN 

STRUC 

THERM 

TTC 

I  OUA LS 

1.00000 

>  APOGEE 

.*1358 

1.00000 

design 

* . 5020# 

.30533  1.00000 

* 

STRuC 

.00075  -.22437 

1.00000 

.1*700 

■  :5m 

-.10494 

1.00000 

TTQ 

.2*707 

•53582  -.10850 

-.12345 

.50935 

1.00000 

1-Ta1L£D 

SIG.  Of  CORRELATION  MATRIXi 

'  .  '  IS 

printed  a op 

DIAGONAL  elements. 

AC31 

ACS  2 

ACSS 

AKHL 

AKM2 

COMM 

EPS1 

EPS2 

EPS3 

ACSl 

*CZ2 

.0570* 

AC  zs 

.nisi 

.02239 

. 

AKM1 

.nuo 

.1*07* 

.2«*30 

AKM2 

.14371 

.29419 

.217*3 

.00000 

f  COMM 

.030*9 

.04*4* 

.055*3 

.39959 

.22599 

1,  E2S1 

.39772 

.17738 

.14943 

.1*452 

.19427 

.*5590 

S  eps: 

.  15795 

.*3534 

.03140 

.0*549 

.04940 

.12599 

.31503 

*"  o>ss 

.  00515 

.00553 

.12472 

.30599 

.*7127 

.  C0891 

-IS*»« 

.13081 

J  lire 

.0241* 

.27281 

.0*3*3 

.35714 

.31**5 

.00033 

.*2254 

.20593 

.0741* 

^  NHAAO 

.28*41 

.*3218 

.397*1 

.2*300 

.20743 

.2041* 

.*2*1* 

.29957 

.1*130 

LAUNCH 

.021*4 

.12521 

.14971 

.*9590 

.30*2* 

.00001 

.2*320 

.13419 

.00090 

QUALS 

.04575 

.***04 

.n*20 

.2*722 

.23375 

.01775 

.*9*19 

.09972 

.05*4* 

jj  APOGEE 

.03900 

.43239 

.0*17* 

.222** 

.29191 

.00019 

.*2091 

.  19579 

.15321 

design 

.*2494 

.25030 

.23333 

.21350 

.20431 

.21404 

.2109* 

.29525 

.22999 

'  STRUC 

.1*455 

.0*037 

.0334* 

.055  1  3 

.19459 

.04*94 

.0000* 

.27925 

.00314 

THERM 

.14787 

,12517 

01502 

.**9*2 

.*212* 

-222*3 

.*7911 

.45455 

.2**77 

ttc 

.00970 

.*7852 

.2*900 

.21591 

.19937 

.29300 

.**019 

.2*392 

.20132 

LIFE 

7MA90 

LAUNCH 

QUALS 

APOGEE 

DCS ION 

STRUC 

THERM 

TTC 

LIFE 

, 

NMARQ 

.24777 

. 

j  LAUNCH 

.00055 

.03413 

, 

,  CUALS 

.07*04 

.07*23 

.12521 

i  u>tca 

.00000 

.21*52 

.00079 

.0**09 

CESIGN 

.04051 

.07757 

.  3*030 

.01497 

.10592 

STRUC 

.*5945 

.*7459 

.0713* 

.15432 

.*5975 

.10319 

therm 

.23304 

.0521* 

.05491 

.27919 

.15151 

.1*974 

.33450 

TTC 

.0130* 

.02417 

.03*:; 

.141*9 

.01039 

.33*12 

.312*3 

.01542 
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At  this  point  the  objective  in  eliminating  variables  became 


to  maximize  the  percentage  of  variance  explained. 12  jn  addition 
to  large  loadings  on  their  principal  factors,  ACS1 ,  TTC  and 
LAUNCH  had  significant  loadings  in  other  columns.  By  trying 
various  combinations  it  was  found  that  by  eliminating  ACS1  and 
TTC  but  retaining  LAUNCH  the  variance  explained  reached  a  maximum 
of  81.7%. 13  £  scree  plot  of  the  eigenvalues  and  the  final 
rotated  factor  matrix  are  shown  in  Figure  2.5  on  page  56. 

Factor  Interpretation 

The  clustering  of  the  variables  and  the  strong  loadings  lead 
quite  easily  to  conclusions  as  to  the  nature  of  the  four  factors 
that  describe  the  technology  embedded  in  a  satellite. 

FACTOR  1  can  be  described  as  MISSION .  To  describe  mission, 
the  requirements  must  be  specified  in  terms  of  APOGEE,  LIFE, 
COMM  and  LAUNCH: 

APOGEE  —  (Orbital  apogee  *  Design  life  *  (%  Quality  S 
+  0.8  Z  Quality  B)  /  10000)  --  Add  to  LIFE  a 
description  of  the  required  apogee  and  quality  levels,  in 
percentages  S  and  B. 

LIFE  --  What  must  the  design  life  of  the  satellite  be? 

COMM  —  While  the  variable  is  actually  required  power 
for  communications  equipment,  it  should  be  easily  estimable 
from  mission  specifications. 

LAUNCH  —  How  will  the  satellite  be  launched? 


I 


VAR ! MAX  ROTATION  1  FOR  EXTRACTION  1  IN  ANALYSIS  \  -  KAISER  NORMALIZATION. 


VAR  I MAX  CONVERGED  IN 


4  ITERATIONS. 


ROTATED  FACTOR  MATRIX: 


FACTOR  l 

FACTOR  2 

FACTOR  3 

FACTOR  < 

APOGEE 

.94321 

.03372 

-.09904 

.12040 

LIFE 

.03274 

.00152 

.04545 

COMM 

.93909 

.13307 

.34424 

.04141 

LAUNCH 

.73759 

.04949 

.42799 

.41991 

AKM2 

.09494 

.99019 

-.04453 

.09201 

AXM1 

.03499 

.94110 

-.20144 

.05145 

ACS2 

.14492 

-.09000 

.90553 

-.32373 

ERS3 

.34379 

.01273 

.79219 

.25341 

ERSi 

-.12943 

-.27471 

.53792 

.07219 

THeRH 

.14552 

- .  09440 

-.15751 

.94439 

NHARO 

.09149 

.21310 

.17057 

.74014 

and  Rotated  Factor  Matrix 
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FACTOR  2  is  an  indirect  measure,  ORBITAL.  The  apogee  kick 
motor  is  used  in  obtaining  the  correct  orbit.  If  the  designer 
knows  the  MISSION,  the  launch  method,  and  the^  required  apogee  and 
shape  of  the  orbit,  rough  specification  of  the  two  AKM  variables 
should  not  be  difficult: 

AKM2  —  Propellent  weight  /  Dry  weight. 

AKM1  —  Specific  impulse. 

FACTOR  3  is  an  indirect  description  of  the  electrical  power 
system  technology. 

EPS3  —  Array  topology. 

EPS1  —  Battery  Capacity. 

Note  that, 

ACS2  —  Primary  stabilization  method, 
is  integrally  related  to  array  topology  due  to  its  implications 
for  array  deployment.  Therefore,  these  three  variables  make  up 
a  single  factor  we  can  label  ELECTRICAL  POWER. 

FACTOR  4  is  a  description  of  the  ENVIRONMENT.  Two 

environmental  variables  load  on  this  factor: 

THERM  —  Max  temperature  -  Min  temperature. 

NHARD  —  Nuclear  hardening. 

The  temperature  range  affects  component  design. ^  The 
required  level  of  nuclear  hardening  is  mission-determined.  The 
most  logical  interpretation  is  ENVIRONMENT  because  these  variable 
values  result  from  the  environment  in  which  the  mission  must  be 


conducted . 


To  recap,  there  are  four  factors  that  describe  the 
technology  embedded  in  a  satellite  and,  together,  account  for 
81.72  of  the  variance  in  the  sample: 


FACTOR 

1 

2 

3 

4 


LABEL 

MISSION 

ORBITAL 

ELECTRICAL  POWER 
ENVIRONMENT 


VARIABLES 

APOGEE,  LIFE,  COMM, 
LAUNCH 

AKM2,  AKM1 

EPS3 ,  ACS2 ,  EPS  1 

THERM,  NHARD 


These  variables  should  be  ascertainable  during  the  early 
decision-making  stages  of  the  system  life-cycle,  thereby  conform 
with  the  requirement  set  in  Dodson's  third  criterion. 


Factor  Scores 


The  next  step  was  to  construct  a  table  of  factor  scores  for 
all  eighteen  systems. The  four  scores  for  each  of  the  18 
systems,  in  A  through  R  order,  are  shown  in  Table  2.1  on  page  59. 

Recall  that  THERM  was  missing  for  System  R.  In  calculating 
the  factor  scores  for  System  R  the  sample  average  value  of  THERM 
was  used. 


ELLIPSOID  CONSTRUCTION 


Dodson's  most  important  contribution  to  the  development  of 
SOA  theory  has  been  the  ellipsoid  method  of  technology 
specification.^  Seen  in  two  dimensions,  the  ellipsoid  model  is 
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Table  2.1  Factor  Scores 


•-  Factor 

Scores  -- 

System 

Fact  1 

Fact  2 

Fact  3 

Fact  4 

A 

0.5481 

1.4894 

1.4514 

1.3038 

B 

-0.7021 

-0.6149 

0.8253 

-0.8771 

C 

1.4523 

-1.1906 

0.3233 

2.3646 

D 

-0.2591 

1.1941 

0.8042 

0.0857 

E 

-0.8919 

-0.7342 

0.4668 

-0.6761 

F 

-0.5195 

0.6648 

-1.3337 

0.8363 

G 

-0.2870 

-1.3133 

-1.7576 

0.8937 

H 

0.4619 

-0.8923 

-0.4650 

-1 .1932 

I 

2.1776 

-1.0966 

0.4696 

-0.3093 

J 

-1.4171 

-1.0918 

1.5999 

-0.0233 

K 

-1.0725 

0.9872 

0.2058 

0.4739 

L 

-0.9641 

-0.6474 

0.0670 

-0.2184 

M 

-0.1629 

-0.8299 

0.2157 

-0.4064 

N 

-1.1595 

0.7442 

-1.0434 

1.0676 

0 

0.6814 

0.9718 

-0.4915 

-1.6992 

P 

0.1755 

0.7231 

-0.4819 

-0.8667 

Q 

0.7529 

0.4861 

-1.7235 

-0.4232 

R 

1.1859 

1.1505 

0.8674 

-0.3126 

relatively  simple 

.  ( See 

Figure  2 

.6  on  page  60.)18  Two  system 

attributes  are  scaled  on 

X 1  and 

X2.  The 

ellipse  represents  a 

particular  level  of  technology  or  SOA.  Point  A  lies  on  the 


ellipse.  The  general  idea  is  that  if  mission  requirements  call 
for  more  Xj  the  need  could  be  met  by  moving  to  the  right  along 
the  curve,  thereby  giving  up  some  X2.  Note  SOA  is  not  increased. 


The  curve  can  be  thought  of  as  an  iso-SOA  curve. 

The  SOA  curve  exhibits  diminishing  returns.  In  order  to 
obtain  more  Xj  ever-increasing  quantities  of  X2  must  be  given  up. 
This  is  a  common  phenomenon  in  design.^ 

Dodson  suggests  that  (in  N-space)  a  new  hypersurface  is 


required  for  each  SOA.  In  this  study,  however,  only  one 


Figure  2.6  Dodson's  Ellipsoid  Model,  Two  Dimensions 

ellipsoid  was  constructed  for  the  entire  sample.  Therefore,  it 
can  be  thought  of  as  a  technology  hypersurface  rather  than 
representing  a  static  S0A.20 

Each  system  plots  in  N-space  as  a  point.  The  proportional 
radial  distance  from  the  origin  represents  the  level  of 
technology  embodied  in  that  particular  system.  Thus,  Point  B  is 
more  technologically  advanced  than  Point  A  (or  any  other  point 
lying  on  the  "average  technology  level"  curve).  The  procedure 
standardizes  the  distance  from  the  origin  to  the  curve  as  one,  so 
D(Wi,  W2)  is  greater  than  one  indicating  it  represents  an  SOA 
higher  than  the  average  technology  for  the  type  of  systems  under 
study,  satellites. 

Factor  scores  result  from  a  calculation  that  uses  variable 
values  expressed  as  standardized  normal  deviates.  Therefore  many 
factor  scores  are  negative.  Obviously  negative  point  values 
cannot  be  used  in  constructing  an  ellipsoid.  The  procedure 


followed  was  to  add  an  arbitrary  constant,  three,  to  each  factor 
score,  thereby  looking  at  values  from  a  point  three  standard 
deviations  below  the  mean  rather  than  from  the  mean.  Note  that 
factor  scores  represent  a  particular  blend  of  the  four  parameters 
affecting  the  level  of  technology  in  a  system.  The  technology 
level  itself  is  a  function  of  the  four  scores. 

The  modified  factor  scores  were  entered  in  a  computer 
program  designed  to  produce  ellipsoid  parameters  and  point 
values.^  The  program  is  listed  and  the  relevant  run  is  shown 
over  the  next  three  pages.  The  desired  overall  technology  values 
are  shown  in  the  column  headed  "MU-J",  but  it  is  important  to 
remember  that  each  system  has  coordinates  on  four  factor 
attributes  as  well  as  a  distance  from  the  origin. 

Output 


The  final  equation  for  the  ellipsoid  representing  satellite 
technology  is, 


where , 

X. 

1 

There  is  no  statistically  rigorous  methodology  for 


X1 

+  X2  + 

x2 

X2 

.  4 

6.9A462 

+  2  + 

6 . 2728z 

6.13382 

+  2 
7.3472^ 

is  the  factor  score  (plus  three)  for  the  ith  factor 


determining  adequate  sample  sizes  in  ellipsoid  construction.  The 
objective,  of  course,  is  to  achieve  stability  of  the  surface  in 
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Figure  2.7  Ellipsoid  Prograa  (Continued) 
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Figure  2.8  Ellipsoid  Program  Output 
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hyperspace.  (An  intuitive  definition  of  stability  is  that  little 
change  in  the  positioning  of  the  ellipsoid  would  result  from 
adding  another  data  point.)  The  author  has  constructed 
ellipsoids  from  a  number  of  data  bases  and,  while  there  is 
obviously  some  variation  from  case  to  case,  reasonable  stability 
is  usually  achieved  when  the  number  of  observations  in  the  sample 
is  at  least  four  times  the  number  of  variables  per  observation. 
This  rule  of  thumb  is  recommended  as  a  guideline.  In  the  case  at 
hand  there  were  four  factors  and  18  systems,  so  the  sample  size 
guideline  was  met. 


ANALYTICAL  SUMMARY 

Before  discussing  the  results  it  might  be  useful  to 
summarize  the  steps  taken: 

1.  The  85  variables  used  by  the  Air  Force  to  describe 
satellite  technology  were  studied  by  engineers  to  separate  those 
which  are  design  objectives  from  those  which  are  simply 
byproducts  of  the  design.  The  result  was  18  relevant  variables 
or  composite  variables  constructed  so  as  to  observe  Dodson's 
three  criteria. 

2.  Factor  analysis  was  used  to  take  advantage  of 
redundancies  in  the  data,  thereby  clustering  variables  onto  four 
"factors".  Thus,  the  informa  ti on  content  of  18  technology 
describers  was  condensed  down  to  four  factor  scores  for  each 


satellite. 


3.  The  factor  scores  formed  the  basis  for  determining  a 
technology  ellipsoid  for  satellites.  The  point  position  of  each 
individual  satellite  was  used  to  calculate  a  proportional 
distance  from  the  origin,  which  is  a  single  measure  of  SOA. 

The  steps  are  summarized  in  Figure  2.9  below: 

Input  »»>  [Process]  *■>  Output 

85  variables  »■>  [expert  judgement]  »■>  18  relevant  variables 

18  relevant  variables  — >  [factor  analysis]  ■»>  4  factor  scores 

4  factor  scores  ■■>  [ellipsoid  method]  -■>  A  single  technology 

measure  containing 
much  information 

Figure  2.9  Summary  of  Analytic  Steps 

Results 

The  validity  of  this  method  can  be  seen  in  several  ways. 
First,  consider  a  chronological  plot  of  SOA  by  satellite.  (See 
Figure  2.10  on  page  67.)  The  dates  shown  on  the  horizontal  axis 
in  the  graph  are  the  years  in  which  the  18  satellites  studied 
were  first  launched.  The  SOA  measure  is  on  the  vertical  axis. 
For  several  years,  from  1966  through  1977,  satellite  technology 
remained  relatively  constant.  Some  particular  systems  were  more 
"advanced"  than  others,  but  the  SOA  was  essentially  fixed.  Then, 
in  about  1978,  SOA  began  rising.  The  overall  picture  this 
presents  is  clearly  consistent  with  a  "new"  technology  just 
beginning  to  enter  the  accelerating  portion  of  its  S-curve.^ 
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Figure  2.10  Satellite  SOA  Through  Time 


Another  way  to  examine  the  growth  in  satellite  technology 
during  the  twenty  years  represented  is  to  use  Leinhard's  method 
to  calculate  the  growth  rate  and  to  compare  the  observed  growth 
rate  for  satellites  with  that  of  other  technologies.^ 

Begin  by  plotting  ln(SOA)  against  time.  (See  Figure  2.11  on 
page  68.)  A  regression  fit  to  this  data  form  is  highly 
significant  (but  does  have  a  rather  large  standard  error).  The 
equation  is. 
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Figure  2.11  Log  form  of  SOA  Through  Time 
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Lienhard's  equation  shows  satellite  technology  has  grown  at 
a  30-year  n-folding  of  3.71:  the  "time  constant,"  T  is  22.9 
years  for  an  enfolding.  This  means  satellite  technology  is 
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growing  raster  than  the  thermal  efficiency  of  steam  power  did 
from  1742-1850,  but  somewhat  more  slowly  than  the  speed  of  air 
transport  from  1884-1967. 

It  is  also  instructive  to  examine  the  breakdown  of  scores  on 
the  individual  factors  through  time.  The  following  graph  looks 
at  three  time  intervals,  1966-74,  1975-77  and  1978-83,  plotting 
the  four  average  (by  satellite)  factor  scores  during  each. 

•  The  average  SOA  measure  for  the  1966-74  systems  was  0.715; 
the  average  for  1975-77  was  insignificantly  higher,  0.751.  But 
the  composition  of  factor  scores,  or  the  coordinates  of  the 
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Figure  2.12  Factor  Score  Growth,  1966-1983 
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representative  points  on  the  ellipsoidal  hypersurface  are 
different.  During  1975-77  considerably  more  emphasis  was  placed 
on  advanced  electrical  power  systems,  Factor  3. 

The  satellites  launched  during  1978-83  were  more  advanced  in 
all  four  factors,  and  the  average  SOA  measure  was  considerably 
higher,  1.179.  The  one  satellite  launched  in  1986  is  not 
included  in  the  last  group,  but  its  SOA  was  even  higher,  1.643. 
It  's  score  on  the  mission  factor  was  comparable  to  the  1978-83 
group,  but  it  was  considerably  more  advanced  in  the  orbital , 
electrical  power  and  environment  factors. 

Still  another  way  to  view  the  progress  in  SOA  is  to  look  at 
the  average  level  embodied  in  all  satellites  launched  each  year. 
That  approach  is  taken  in  the  graph  below.  (See  Figure  2.13  on 
page  71.)  Again,  we  see  the  upward  curving  shape  so  typical  of 
the  low  end  of  an  S-curve. 

Chapter  Objective  Revisited 

The  objective  of  this  Chapter  was  to  provide  technology 
measures  that  will  be  useful  in  determining  development  cost. 
Operationally,  if  one  knows  the  status  of  technology  at  the  onset 
a  development  project,  and  the  desired  design  characteristics 
-i*  new  system  to  be  developed,  one  (hopefully)  will  be  in  a 
i  to  estimate  the  development  cost  to  be  incurred. 

•  n  will  be  addressed  in  the  next  chapter.  This 
the  SOA  measures  to  be  tested. 
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17.  See  op.  c  it  .  ,  "A  General  Approach  to  Measurement  of  the 
State  of  the  Art  and  Technological  Advance,"  Technological 
Forecasting .  Vol.  1,  (1970),  pp.  391-408. 

18.  This  figure  is  taken  from  Dodson,  ibid . ,  p.  400. 

19.  This  property  is  described  in  more  rigorous  terms  by 
Grosch's  Law.  See  Knight,  K.  E.,  "A  Functional  and  Structural 
Measurement  of  Technology,"  Technological  Forecasting  and  Social 
Change.  27,  (1985),  pp.  107-127. 
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written  in  FORTRAN  and  was  based  in  part  on  a  matrix  inversion 
subroutine  contained  in  McCormick,  J.  M.  and  M.  Salvador i , 
Numerical  Methods  in  FORTRAN  (New  York:  Prentice-Hall,  1964), 
Program  9-) .  The  program  was  recoded  by  the  author  in  BASIC  for 
an  IBM  PC. 
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22.  A  good  treatment  of  S-curves  as  a  method  of  measuring  SOA 
advance  can  be  found  on  pages  18-24  of  Gordon,  T.  J.  and  T.  R. 
Munson,  "A  Proposed  Convention  for  Measuring  the  State  of  the  Art 
of  Products  or  Processes",  Technological  Forecasting  and  Social 
Change.  Vol.  20  (1981),  pp.  1-26. 

23.  Lienhard,  J.  H.  ,  "The  Rate  of  Technological  Improvement 

before  and  after  the  1830s,"  Technology  and  Culture,  (July  1979), 
pp  .  5  15-530  .  See  especially  pp.  516-517,  but  note  that 

Leinhard's  Equation  (1)  contains  typos.  For  a  correct  version 
of  his  Equation  (1)  see  page  29  of  this  report. 


Chapter  3 


COST  ESTIMATION  AND  CONTROL 

INTRODUCTION 

The  end  purpose  of  developing  SOA  measures  is  to  expedite 
prediction  of  the  cost  of  developing  new  technological  systems, 
which  is  the  initial  step  in  any  attempt  to  control  such  costs. 
It  is  therefore  fitting  to  search  for  statistical  associations 
(if  any)  between  the  degree  to  which  a  system's  technology  has 
been  extended  through  successive  development  efforts  and  the 
level  of  activity  required  to  bring  these  extensions  about. 

SOA  EXTENSION  CONCEPTS 

The  definition  and  measurement  of  the  amount  of  change 
represented  by  a  particular  technological  development  requires 
careful  consideration.  In  Chapter  2  the  level  of  technology 
embodied  in  each  of  18  satellites  was  determined,  and  each  was 
expressed  in  terms  of  a  point  in  hyperspace,  but  the  concept  of 
an  SOA  extension  was  not  articulated. 

An  SOA  extension  involves  the  movement  of  technology  from 
one  point  to  another.  If  at  the  outset  of  a  particular 
development  project  the  technological  objective  of  the  project 
can  be  specified  as  a  point  in  hyperspace,  and  if  the  closest 


existing  technology  can  also  be  identified  as  a  point,  then  the 
development  task  at  hand  can  be  defined  by  referencing  the 
technological  distance  separate  ■*  the  two.*  This  is  simply  the 
Euclidian  distance  between  the  's. 

Consider  first  a  mathematics  ement  of  technological 

distance: 

d.  . 

where, 

d_  ■  the  technological  distance  between  the  ith’  and  j_th 
systems , 

S.k  »  a  technological  attribute  score  of  the  it h  system  on 
the  kth  attribute, 

Sjk  »  a  technological  attribute  score  of  the  j_th  system  on 
the  kth  attribute, 

N  -  the  number  of  technological  attributes  used  to  define 
SOA  for  systems  of  this  type. 

Several  more  detailed  measurement  concepts  will  now  be 
discussed.  These  ideas  are  most  easily  illustrated  in  two 
dimensions,  but  they  are  clearly  generalizable  to  N-dimension 
settings . 

Consider  the  graph  shown  in  Figure  3.1  on  page  77.  In  1985 
Martino  measured  the  technology  embodied  in  numerous  aircraft 
designs  by  making  use  of  three  indexed  variables,  structural 


What  is  the  technological  distance  to  be  spanned  by  this 
development  task?  Geometrically,  it  would  be  the  minimum  radial 
distance  to  the  P-30  from  some  other  currently  existent  system. 
By  calculating  technological  distances  we  can  easily  conclude 
that  the  "closest"  predecessor  technology  is  represented  by  the 
P-26;  that  distance  can  be  symbolized  as  P-26  — >  P-30. 

Now  consider  three  other  distinct  measures  of  technological 
spread.  Designate  these  reach ,  advance  and  redesign . 

Definitions 

The  reach  of  the  P-26  is  its  distance  from  the  origin, 
easily  calculated  from  the  formula  for  technological  distance  by 
setting  3jk  =  0.  Similarly,  the  reach  of  the  P-30  is  the 
distance,  Origin  — >  P-30. 

The  advance  represented  by  moving  from  the  P-26  to  the  P-30 
is  (Origin  — >  P-30)  -  (Origin  — >  P-26),  or  the  difference  in 
the  reach  of  the  two  systems,  also  depicted  as  A  — >  P-30. 

Finally,  redesign  is  the  lateral  shift  associated  with  the 
changing  blend  of  technological  attributes  between  the  two 
aircraft,  P-26  — >  A.  This  distance  can  be  approximated  as. 


which  is  just  the  square  root  of  the  technological  distance 
between  the  P-26  and  the  P-30  squared  less  the  advance  squared 
(by  simple  Euclidian  geometry). 


HYPOTHESES 


The  first  hypothesis  to  be  tested  is  that  the  difficulty  of 
the  development  task,  as  measured  by  the  time  required  for  its 
completion .  is  a  function  of  the  three  measures  of  technological 
spread.  The  advance  represents  the  "invention”  aspects  of  the 
de  ve lo pm en t--t he  "true"  SOA  progress  required.  The  redesign 
portion  represents  a  different  (and  perhaps  less  demanding) 
aspect  of  the  project — movement  parallel  to  an  old  SOA  surface. 
(Note  that  advance  and  redesign  can  be  thought  of  as  the  vector 
components  of  the  technological  distance  -between  the  two 
systems.)  Finally,  r  ea  ch  measures  the  total  technological 
complexity,  or  the  overall  ambition  demanded  by  the  project. 

H^:  Development  Time  =>  f(Advance,  Redesign,  Reach) 

Relevant  Satellite  Data 


The  data  relevant  to  this  portion  of  the  analysis  are  shown 
in  Table  3.1  on  page  80.  The  reported  cost  figures  are 
"nonrecurring"  development  costs  provided  by  the  Air  Force  for 
each  system,  adjusted  to  constant  1986  dollars  by  using  the  OSD 
3600  Escalation  Index  of  prices  for  development  work.  The  "time" 
column  reports  the  time  elapsed,  in  months,  from  awarding  the 
development  contract  to  the  first  launch  of  the  satellite.  The 
figures  for  advance,  redesign  and  reach  were  calculated  as 
formulated  above  from  the  four  factor  scores  reported  in' 
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Table  3.1  Data  for  Hypothesis  Testing 


Min  Dist 

NR  Cost 

Devel 

System 

Predecessor 

FY86$ 

Time(Mo) 

Advance 

Redesign 

Reach 

H 

G 

73594.4 

25 

0.01947 

0.37529 

0.76986 

0 

F 

116580.0 

27 

0.02398 

0.41191 

0.93837 

L 

H 

37228.4 

32 

0.01076 

0.26210 

0.78737 

J 

H 

155522.8 

46 

0.10584 

0.44910 

0.93328 

N 

F 

32585.5 

28 

0.00719 

0.10878 

0.91366 

M 

H 

319498.7 

37 

0.03647 

0.17506 

0.82829 

B 

H 

91707.4 

37 

0.05194 

0.27101 

0.85233 

Q 

F 

64383.3 

37 

-0.01799 

0.25978 

0.87590 

P 

0 

121932.3 

33 

-0.02440 

0.13829 

0.90234 

E 

L 

108084.9 

37 

0.00867 

0.09139 

0.80131 

K 

N 

14943.2 

36 

0.05180 

0.21674 

0.98883 

R 

P 

180652.8 

56 

0.18031 

0.21766 

1.15432 

C 

M 

157820.4 

64 

0.22497 

0.38644 

1 . 14932 

I 

H 

451274.0 

86 

0.18782 

0.25399 

1.04977 

Table  2.1  on  page  59  (plus  the  constant,  3^  to  position  all 
values  in  positive  space). 

Four  satellites  were  excluded  from  Table  3.1,  and  from  this 
stage  of  the  analysis.  Systems  F  and  G  were  omitted  because  they 
had  no  chronological  predecessors.  That  is,  no  satellites  from 
the  available  data  set  had  been  launched  prior  to  F  and  G's 
contract  award  dates.  Satellite  D  was  omitted  because  its 
contract  award  date  was  unknown.  Finally,  A  was  omitted  because 
it  was  known  to  have  been  a  very  minor  upgrade  of  another 
satellite  for  which  data  were  not  available.  The  data  were 
complete  for  the  remaining  14  systems,  except  that  only  the  year 
of  contract  award,  not  the  month,  was  known  for  systems  R,  C 
and  I.  In  each  case  the  contract  was  awarded  in  1976.  System  K 

was  also  contracted  in  1976 — in  February.  The  working  assumption 

■» 
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adopted  was  to  treat  R,  C,  and  I  as  being  awarded  in  February  as 
well,  but  other  assumptions  were  analyzed  for  result  sensitivity 
with  very  little  disparity. 


Test  of  First  Hypothesis 

To  test  the  first  hypothesis,  a  multiple  regression  was  run. 
The  result  was, 

Time  *  52.86  +  218.93  Advance  -  34.28  Redesign  -  17.37  Reach 


t  statistics 

(3.69)  (1.45) 

(0.47) 

Significance 

.001  .085 

.322 

Variance 

2 

explained  (R  ) 

.791 

Adjusted 

R2 

.728 

Standard 

error  of  the  estimate 

8.745 

Taken  as  a  whole,  the  regression  is  highly  significant.  Advance 
is  by  far  the  most  important  determinant  of  development  time. 
Neither  redesign  nor  reach  is  statistically  significant.-^ 
Table  3.2  and  Figure  3.2  on  page  82  show  the  residuals  and  a  plot 
of  predicted  versus  actual  time. 

Development  cost  may  not  be  a  smooth  function  of  development 
time.  As  a  program  drags  on  beyond  its  intended  completion  date, 
it  may  be  relatively  more  costly  to  compress  the  required 
accomplishment  into  an  increasingly  abbreviated  time  horizon. 
Putnam  has  humorously  but  graphically  illustrated  the  difficulty 
of  trying  to  hurry  development  projects  by  adding  more  people  in 


Table  3.2  Residuals  from  the  Time  Regression 


ACTUAL 

Tue 

PREDICTED 

Ti*e 

RES i DUAL 

nc 

30.3? 

-5.39 

27 

27.  a? 

-.6? 

_  w 

32.36 

-.56 

28 

34.84 

-6.84 

46 

44.43 

1.57 

37 

40.46 

-3.46 

37 

40.14 

-3. 14 

37 

24.8 

12.2 

vJ 

27.11 

5.89 

37 

37.7! 

-.71 

36 

3?. 6 

*3.6 

36 

04.a3 

-8.83 

64 

c.a.9i 

-4.91 

36 

67.04 

18.96 

Development  Time 


Quality  of  Model 


0  20  40  60 

Predicted  Time 

Figure  3.2  Predicted  versus  Actual  Development  Time 


The  implication  is  that  there  may  be  a  "natural 


his  Figure  3.* 
development  time"  for  SOA  extension  projects. 
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INTERPRETATION  OF  RESULTS  FOR  CONTROL 


The  most  basic  principle  of  cost  control  is  to  attribute 
differences  between  expected  costs  and  actual  costs  to  causes. 
That  is,  to  explain  variances  between  predicted  and  actual  costs. 
The  preceding  analysis  affords  us  an  opportunity  to  do  exactly 
that . 


Consider  the  information  contained  in 
column  labeled  ”Ex  Ante  Cost  E s t "  was 
following  way. 

1.  The  particular  system’s  values 
and  Reach  (all  determinable  ex  ante)  were 
regression  to  predict  a  time  that  would 
system's  development. 


Table  3.3  below.  The 
constructed  in  the 

for  Advance,  Redesign 
entered  in  the  "Time'’ 
be  required  for  the 


Table  3.3  Calculations  for  Performance  Variances 


Ex  Ante 

Cost  Est 

Variance 

Cost 

Cost 

Based  on 

Due  to 

Actual 

Control 

Total 

System 

Est 

Act  Time 

Time 

Cost 

Variance 

Variance 

H 

86693.6 

43171.8 

-43521.8 

73594.4 

30422.6 

-13099.2 

0 

71378.8 

66256.4 

-5122.4 

116580.0 

50323.6 

45201.2 

L 

94692.3 

90575.8 

-4116.5 

37228.4 

-53347.4 

-57463.9 

J 

105629.1 

55081.5 

-50547.6 

32585.5 

-22496.0 

-73043.6 

N 

151593.0 

163210.9 

11617.9 

155522.8 

-7688.1 

3929.8 

M 

132567.0 

107001.7 

-25565.2 

319498.7 

212497.0 

186931.7 

B 

131031.5 

107834.1 

-23197.4 

91707.4 

-16126.7 

-39324.1 

Q 

57540.8 

147671.9 

90131.1 

64383.3 

-83288.6 

6842.5 

P 

68571.3 

112126.8 

43555.5 

121932.3 

9805.5 

53361.0 

E 

119390.3 

114144.5 

-5245.8 

108084.9 

-6059.6 

-11305.4 

K 

128438.5 

101848.3 

-26590.3 

14943.2 

-86905.1 

-113495.3 

R 

249372.7 

184120.6 

-65252.1 

180652.8 

-3467.8 

-68719.9 

C 

268926.2 

232652.4 

-36273.8 

157820.4 

-74832.0 

-111105.8 

I 

259983.1 

400111.4 

140128.3 

451274.0 

51162.6 

191290.9 

2.  The  Predicted  Time  was  input  to  the  "Cost"  regression, 
with  the  Residual  set  to  zero  (thereby  assuming  the  predicted 
time  will  be  achieved),  to  calculate  the  ex  ante  prediction  of 
development  cost. 

Next  the  actual  time  for  the  project  was  compared  to  the 
predicted  time  to  determine  the  Residual  (ex  post).  The  Cost 
regression  was  then  revisited  with  values  for  both  variables,  and 
a  new  cost  estimate  constructed,  c  on  si de  ring  the  actual  time 
residual  for  the  project.  This  produced  the  "Cost  Est  Based  on 
Act  Time"  column. 

The  difference  between  the  ex  ante  cost  estimate  and  the 
cost  estimate  based  on  the  project's  actual  time  has  been  termed 
the  "Variance  Due  to  Time".  This  figure  is  a  best  estimate  of 
the  portion  of  the  total  variance  that  can  be  attributed  to  the 
cost  consequences  of  time  delays  (or  to  fortuitus  and  perhaps 
unforseen  acceleration  of  the  schedule).  Minus  figures  are 
favorable,  positive  are  unfavorable. 

When  Actual  Cost  is  compared  with  the  cost  estimate  based 
on  actual  time  the  result  is  a  "Coat  Control  Variance".  Given 
that  the  project  actually  took  t_  time  unita  to  complete,  the  cost 
should  have  been  "Cost  Est  Based  on  Act  Time".  The  actual  cost 
wa s  a  different  amount,  so  the  variation  is  attributed  to  cost 
control.  Again,  minus  figures  are  favorable. 

The  variances  identified  above  have  been  shown  graphically 
in  Figures  3.5  and  3.6  on  pages  87  and  88.  A  graph  of  Total  Cost 
Variance  appears  on  page  89. 
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Cost  Variance  due  to  Time 

PasiO>«  Variance*  ore  Unfavorable 


Figure  3.5  Time  Variances  for  Satellite  Development  Cost 


Aa  Figure  3.7  indicates,  there  were  significant  unfavorable 
Total  Cost  Variances  (expressed  as  a  percent  of  estimated  cost) 
for  only  four  of  the  14  progress.  The  cause  of  the  variance  in 
two  of  these  cases,  progress  P  and  I,  appears  to  be  timing 
problems;  in  the  other  two  cases,  'programs  0  and  M,  the  origin 
of  the  problem  seems  to  be  cost  control.  (If  these  variances 
appear  large  compared  with  what  the  reader  is  accustomed  to, 
consider  the  fact  that  there  is  far  greater  uncertainty 
surrounding  a  research  project's  costs  than  those  of  a  routine 
manufacturing  operation.  Also,  in  fsirness,  consider  the 
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Figure  3.6  Cost  Control  Variances  for  Satellite  Programs 

imprecise  nature  of  the  data  available  for  this  study!) 

Significant  favorable  Total  Cost  Variances  were  experienced 
in  three  cases,  programs  L,  J  and  K.  These  pleasing  results  can 
be  attributed  to  fortuitus  timing  in  the  case  of  Program  J,  and 
to  excellent  coat  control  in  programs  L  and  I. 

Program  Q  had  an  interesting  outcome;  the  time  required  for 
development  was  much  greater  than  expected,  so  the  cost  variance 
due  to  time  is  large  and  unfavorable,  but  a  very  favorable  cost 
control  variance  offset  the  timing  effects — leaving  the  total 
cost  variance  insignificant. 


Figure  3.7  Total  Cost  Variances  for  Satellite  Programs 


CHAPTER  SUMMARY 

This  chapter  presented  the  fundamentals  of  estimating 
development  cost,  once  the  proper  technology  measures  are  known 
for  the  both  the  system  under  development  and  an  appropriate  base 
of  predecessor  systems.  It  has  been  shown  that  development  time 
estimation  is  a  critical  intermediate  step  in  this  process. 

In  addition,  methods  for  identifying  and  quantifying  cost 
variances  have  been  illustrated.  This  capability  can  provide 
information  that  may  be  useful  for  cost  control. 


ENDNOTES 


1.  See  Dodson,  E.  N.  ,  "Measurement  of  State  of  the  Art  and 
Technological  Advance,"  Technological  Forecasting  and  Social 
Change ,  27,  (1985),  pp.  129-146.  The  concept  of  a  Technological 
Distance  Score  and  the  equation  that  follows  are  discussed  on 
pp.  141-142. 

2.  Martino,  J.  P.,  "Measurement  of  Technology  Using  Tradeoff 
Surfaces,"  Technological  Forecasting  and  Social  Change.  27, 
(1985),  147-160.  See  especially  pp.  153-157. 

3.  The  model  performs  almost  as  well  if  the  reach  and  redesign 
variables  are  omitted--R^  drops  only  to  0.747  and  the  SEE  rises 
to  8.786.  The  full  model  is  retained  because  of  its  slightly 
better  overall  fit. 

4.  This  illustration  was  taken  from  Putnam,  L.  H.,  Software 
Cost  Estimating  and  Life-Cycle  Control.  The  Institute  of 
Electrical  and  Electronics  Engineers,  Inc.,  Computer  Society 
Press,  (1980),  p.  99. 
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SUMMARY,  CONCLUSIONS  AND  RECOMMENDATIONS 


INTRODUCTION 

When  the  various  components  of  the  Department  of  Defense 
(DoD)  enter  into  contractual  agreements  calling  for  extensions  of 
the  State-of-the-Art  (SOA),  uncertainty  generally  requires  that 
some  variant  of  cost-plus  contracting  be  employed.  Therefore,  it 
is  important  that  DoD  possess  a  highly  developed  ability  to 
estimate  the  likely  cost  of  achieving  the  desired  technological 
advance . 

This  study  examined  the  techniques  currently  available  for 
costing  SOA  extension  contracts  by  surveying  the  literature 
dealing  explicitly  with  SOA  measurement  and-  costing.  There  was 
general  recognition  that  the  cost  of  an  SOA  extension  relates  to 
the  scale  of  the  undertaking.  Most  of  the  research  that  has  been 
done  to  date  therefore  concentrates  on  measuring  the  amount  of 
SOA  extension  represented  by  a  sample  set  of  efforts.  SOA 
measurement  strategies  usually  employ  regression  to  combine 
design  variables  in  an  effort  to  produce  a  single  measure  of  the 
SOA  for  a  given  system.  The  typical  dependent  variable  is  the 
"year  of  technology". 

The  relationship  between  the  scale  of  an  SOA  extension  and 
development  cost  has  also  been  studied,  but  the  findings  from 


these  studies  have  been  disappointing.  Cost  estimates  made  early 
in  the  life  of  a  project  have  been  plagued  with  error.  The  focus 
of  this  study  has  therefore  been  to  develop  a  cost  estimating 
model  that  is  demonstrably  workable  for  both  prediction  and  cost 
control  uses. 

IMPORTANT  BACKGROUND  LITERATURE 

Three  prior  works  were  found  to  be  of  pivotal  importance. 
These  were, 

1.  Dodson  and  Graver's  development  of  the  ellipsoid  method 
for  measuring  the  level  of  technology  embodied  in  multifarious 
systems:  the  theoretical  correctness  of  this  model  in  its 
second-order  form  has  been  tested  and  demonstrated  in  many 
contemporary  studies.1 

2.  Gordon  and  Munson's  suggestion  that  factor  analysis  is 
an  appropriate  methodology  for  capturing  the  influence  of  a 
relatively  large  number  of  technological  parameters:  this  method 
enables  the  analyst  to  avoid  losing  the  influence  of  vital 
determinants  of  technology. 2 

3.  Alexander  and  Nelson's  definition  of  "technological 
continuity":  their  work  demonstrated  the  need  for  using  data  for 
unbroken  generations  of  systems  when  studying  the  methodologies 
for  measuring  SOA.^ 

The  author's  work  drew  heavily  on  the  background  laid  by 
these  three  efforts.  All  were  of  great  value. 


THE  MEASUREMENT  OF  TECHNOLOGY 


Variable  Selection 

In  selecting  the  variables  to  be  used  to  measure  the 
technology  embedded  in  the  systems  to  be  studied,  there  is  no 
substitute  for  expert  judgement.  Therefore,  technical  expertise 
must  be  sought,  as  it  was  in  this  study.  In  selecting  variables, 
the  experts  must  bear  in  mind  that  the  S  0  A  -  d  e  t  e  r  o  i  n  i  n  g 
characteristics  must  be  at  least  partially  influenced  by  the 
engineering  development-  decisions.  They  should  choose 
characteristics  which  are  goa Is  of  the  design  process.  In 
addition,  the  variables  should  be  specified  so  that  increasing 
values  correspond  to  greater  technical  difficulty.  Finally,  the 
variable  values  should  be  ascertainable  during  the  early 
decision-making  stages  of  the  system  life-cycle. 

The  result  of  this  process  in  the  present  study  was 
identification  of  18  variables  or  composite  variables  that  could 
be  used  to  describe  satellite  technology.  These  18  variables 
were  determinable  for  each  of  the  18  satellites  in  the  sample 
(with  one  minor  exception).  In  general,  no  more  variables  should 
be  specified  than  there  are  systems  in  the  data  set.  Said 
another  way,  there  must  be  at  least  as  many  observations  as  there 
are  relevant  variables. 

This  procedure  worked  well.  There  was  unclouded  discussion 
and  the  results  obtained  represented  consensus. 


Factor  Analysis 

A  factor  analysis  was  then  run.  The  data  were  found  to  be 
quite  robust  with  high  coefficients  of  variation  and  many 
significant  correlations.  In  the  final  analysis,  11  variables 
were  factored.  They  clustered  very  nicely  onto  four  factors  with 
81.72  of  the  variance  explained.  The  factors  were  easy  to 
interpret . 

The  next  step  was  to  calculate  factor  scores  for  the 
systems.  The  factor  scores  are  shown  in  Table  2.1  on  page  59. 

This  process  demonstrated  that  factor  analysis  does  indeed 
provide  a  rigorous  yet  simple  means  for  attaching  considerable 
.uantities  of  information  to  a  very  limited  number  (in  this 
case,  four)  of  variables. 

The  Ellipsoid  Model 


Dodson  suggests  that  (in  N-space)  a  new  hypersurface  is 
required  for  each  SOA.  In  this  study,  however,  only  one 
ellipsoid  was  constructed  for  the  entire  sample.  It  can  be 
thought  of  as  an  average  technology  hypersurface  rather  than  a 
surface  representing  one  static  SOA.^ 

Note  that  the  factor  scores  themselves  represent  particular 
blends  of  the  parameters  affecting  the  level  of  system 
technology.  The  technology  measure,  the  radial  distance  from  the 
origin,  is  a  function  of  the  four  scores. 
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There  is  no  statistically  rigorous  methodology  for 


determining  adequate  sample  sizes  in  ellipsoid  construction. 


However,  the  author's  experience  has  led  him  to  believe  that 


reasonable  stability  is  usually  achieved  when  the  number  of 


observations  in  the  sample  is  at  least  four  times  the  number  of 


variables  per  observation.  This  rule  of  thumb  is  recommended  as 


a  guideline.  Subject  to  this  guideline,  the  method  works  well  in 


providing  system  SOA  measures. 


COST  ESTIMATION  AND  CONTROL 


The  end  purpose  of  developing  SOA  measures  is  to  facilitate 


prediction  of  the  cost  of  developing  new  technological  systems, 


which  is  a  necessary  initial  step  in  any  attempt  to  control  such 


costs.  It  was  therefore  necessary  to  search  for  statistical 


associations  between  (1)  the  degree  to  which  a  system's 


technology  has  been  extended,  and  (2)  the  level  of  activity 


required  to  bring  these  extensions  about. 


More  Precise  Measurement 


The  definition  of  the  change  represented  by  a  particular 


technological  development  required  careful  consideration.  This 


led  to  better  articulation  of  the  concept  of  an  SOA  extension . 


The  technological  objective  of  the  project  and  the  closest 


existing  technology  were  used  to  identify  the  development  task  by 
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referencing  the  technological  distance  separating  the  two. 5  This 
is  the  Euclidian  distance  between  the  two  points. 

Several  more  detailed  measurement  concepts  were  then 
developed.  There  are  three:  reach .  advance  and  redesign.  The 
reach  measures  the  total  technological  complexity,  or  the  overall 
ambition  of  the  project.  Advance  represents  the  "invention" 
aspects  of  the  development--th e  "true"  SOA  progress  required. 
The  redesign  portion  represents  a  movement  parallel  to  an  old  SOA 
surface . 


Testing  the  Time  Hypothesis 


The  first  hypothesis  tested  was  that  the  difficulty  of  the 
development  task,  as  measured  by  the  time  required  for  its 
completion .  is  a  function  of  the  three  measures  of  technological 
spread.  The  result  was. 


Time  ■  52.86  +  218.93  Advance  -  34.28  Redesign  -  17.37  Reach 


t  statistics  (3.69) 


(1.45) 


(0.47) 


Significance  .001 


.085 


.322 


Variance  explained  (R  ) 


.791 


Adjusted  R‘ 


.728 


Standard  error  of  the  estimate 


8.745 


The  regression  is  highly  significant.  Advance  is  by  far  the  most 
important  determinant  of  development  time.  Neither  redesign  nor 
reach  is  statistically  significant. 
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Cost  Prediction  Hypothesis 

It  was  hypothesized  that  development  cost  is  not  a  smooth 
function  of  development  time.  If  a  program  drags  on  beyond  its 
intended  completion  date,  it  becomes  relatively  more  costly  to 
compress  the  required  accomplishment  into  an  increasingly 
abbreviated  time  horizon.  This  suggests  that  there  is  a 
"natural"  project  time,  and  that  the  residuals  from  this  natural 
time  may  influence  cost.  Again,  the  multiple  regression  produced 
good  results, 

Cost  •  -  61357  +  4793.1  Predicted  Time  +  7391.4  Residual 


t  statistics  (3.12)  (2.47) 

Significance  .004  .013 

Variance  explained  (R^)  .590 

Adjusted  R^  .516 

Standard  error  of  the  estimate  82647 


Cost  Control 


The  most  basic  task  in  cost  control  is  to  explain  variances 
between  predicted  and  actual  costs.  The  regressions  developed 
earlier  provided  a  basis  for  doing  so. 

First,  Advance,  Redesign  and  Reach  were  used  in  the  "Tine" 
regression  to  predict  the  time  that  would  be  required  for  the 
system's  development.  Then  the  Predicted  Time  was  input  to  the 
"Cost"  regression  (with  the  Residual  set  to  zero)  to  provide  an 
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ex  ante  prediction  of  development  cost. 

Next  the  actual  time  for  the  project  was  compared  to  the 
predicted  time  to  determine  the  Residual.  The  Cost  regression 
was  then  used  again  —  this  time  to  calculate  a  new  cost  estimate 
considering  the  residual  time  for  the  project.  The  difference 
between  the  ex  ante  cost  estimate  and  the  cost  estimate  based  on 
the  project’s  actual  time  is  the  "Variance  Due  to  Time"  —  the 
portion  of  the  total  variance  that  can  be  attributed  to  the  cost 
consequences  of  time  delays. 

Actual  Cost  is  compared  with  the  cost  estimate  based  on 
actual  time  to  determine  a  "Cost  Control  Variance".  This 
variance  indicates  the  quality  of  cost  control  for  the  project. 

CONCLUSIONS 

The  fundamentals  developed  in  this  study  provide  a  workable 
methodology  for  measuring  the  level  of  technology  embodied  in 
complex  systems,  and  for  measuring  the  degree  of  advance 
represented  by  the  technological  characteristics  of  new  systems 
compared  with  old.  When  a  continuous  data  base  is  available, 
the  methods  in  this  study  have  also  been  shown  to  be  effective  in 
relating  SOA  advance  to  development  cost.  This  capability  can 
provide  information  that  may  be  useful  for  cost  control. 

The  requisite  data  used  in  implementing  this  procedure  have 
a  great  deal  to  do  with  the  quality  of  the  results  achieved.  If 
the  benefits  of  good  cost  control  are  to  be  obtained  it  is 


I 
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absolutely  essential  to  have  a  relevant  and  complete  data  base. 
Let  us  now  turn  to  a  description  of  the  data  requirements. 

Data  Requirements 


1.  Data  must  be  of  sufficient  quantity.  It  is  necessary 
to  have  at  least  four  valid  observations  (systems)  in  the  data 
base  for  every  factor  that  is  identified  through  the  factor 
analysis  procedure.  More  observations  help  to  provide  stability 
in  constructing  the  ellipsoid.  The  data  base  must  contain  no 
fewer  than  16-20  valid  systems. 

2.  The  number  and  quality  of  variables  describing  the 
technology  in  each  system  must  be  sufficient  to  allow  experts  to 
construct  a  reasonably  complete  menu  of  design  objectives.  In 
this  study  there  were  85  variables:  the  engineers  condensed  this 
list  to  18. 

3.  Reliable  development  cost  and  time  data  must  be 
available.  Cost  figures  must  be  dated  for  price-level  adjusting 
and  development  times  must  be  recorded.  There  is  currently  no 
reliable  methodology  for  cost  estimating  that  does  not  make  use 
of  development  time. 

4.  The  size  and  continuity  of  a  data  base  can  be  improved 
by  recording  at  the  sub-system  rather  than  system  level.  For 
example,  missile  and  torpedo  guidance  systems  nay  be  evolutions 
of  one  another.  By  recording  data  for  guidance  systems  rather 
than  for  torpedoes  the  quality  of  a  data  base  is  improved. 
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RECOMMENDATIONS 


Inasmuch  as  the  cost  control  objectives  are  worthwhile  and 
the  research  methodology  has  been  shown  to  perform,  it  would  seem 
desirable  to  attempt  applications  in  settings  other  than 
satellite  development.  The  principal  difficulty  encountered  in 
doing  so  will  undoubtedly  be  the  availability  of  data.  The  most 
significant  recommendation  that  could  be  made,  therefore,  is  that 
serious  efforts  be  made  to  create  and  maintain  relevant  data 
bases  on  major  systems  that  are  expected  to  be  advanced  beyond 
their  present  SOA.  The  data  should  include  technical  information 
as  well  as  cost  and  development  time  details. 

Of  course,  the  methodology  developed  here  can  be  improved, 
and  it  is  recommended  that  efforts  continue  in  that  direction. 
Two  additional  arenas  for  study  come  to  mind — aircraft  and 
electronics.  A  study  of  aircraft  development  cost  could  be 
pursued  with  the  cooperation  of  a  large  manufacturer. 
Electronics  would  require  more  industry-wide  representation,  so 
would  be  a  more  difficult  undertaking. 

Finally,  the  financial  and  operating  characteristics  of 
contractors  who  have  enjoyed  unusual  success  in  cost  control  may 
be  detectable.  If  so,  this  information  could  be  extravagantly 
useful  in  Identifying  and  choosing  among  alternate  sources  for 
new  systems.  The  potential  savings  would  be  so  large  that  this 
question  is  clearly  worth  investigating. 
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ENDNOTES 


1.  See  Dodson,  E.  N.,  "A  General  Approach  to  Measurement  of  the 
State  of  the  Art  and  Technological  Advance,"  Technological 
Forecasting.  1  (1970),  pp.  391-408.  The  issue  of  theoretical 
correctness"  is  demonstrated  by  the  method's  compliance  with 
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Change.  27  (1985),  pp.  107-1277 

2.  See  Gordon,  T.  J.,  and  T.  R.  Munson,  "A  Proposed  Conventio 
for  Measuring  the  State  of  the  Art  of  Products  or  Processes, 
Technological  Forecasting  and  Social  Change.  20  (1981),  pp.  1-26. 

3.  Alexander,  A.  J.,  and  J.  R.  Nelson,  "Measuring  Technological 
Change:  Aircraft  Turbine  Engines,"  Report  No.  R-1017-ARPA/PR ,  The 
RAND  Corporation,  (Santa  Monica,  June,  1972),  37  pgs. 

4.  Bear  in  mind  that  the  only  purpose  of  the  ellipsoid  is  to 
attach  relative  distance-f rom-the-origin  values  to  the  various 
systems  in  the  sample.  In  any  case,  this  representation  is 
allowable  as  a  result  of  the  assumption  of  continuity.  See 
Alexander  and  Nelson,  op .  cit . 


5.  See  Dodson,  E.  N.,  "Measurement  of  State  of  the  Art  and 
Technological  Advance,"  Technological  Forecasting  and  Social 
Change ,  27,  (1985),  pp.  1 29-14<> .  The  concept  of  a  Technological 
Distance  Score  is  discussed  on  pp.  141-142. 
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